Facial action unit detection with local key facial sub-region based multi-label classification for micro-expression analysis

Liangfei Zhang, Ognjen Arandjelovic, Xiaopeng Hong

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Micro-expressions describe unconscious facial movements which reflect a person's psychological state even when there is an attempt to conceal it. Often used in psychological and forensic applications, their manual recognition requires professional training and is time consuming. Therefore, achieving automatic recognition by means of computer vision would confer enormous benefits. Facial Action Unit (AU) is a coding of facial muscular complexes which can be independently activated. Each AU represents a specific facial action. In the present paper, we propose a method for the challenging task that is the detection of activated AUs when the micro-expression occurs, which is crucial in the inference of emotion from a video capturing a micro-expression. This specific problem is made all the more difficult in the light of limited amounts of data available and the subtlety of micro-movements. We propose a segmentation method for key facial sub-regions based on the location of AUs and facial landmarks, which extracts 11 facial key regions from each sequence of micro-expression images. AUs are assigned to different local areas for multi-label classification. Considering that there is little prior work on the specific task of detection of AU activation in the existing literature on micro-expression analysis, for the evaluation of the proposed method we design an AU independent cross-validation method and adopt Unweighted Average Recall (UAR), Unweighted F1-score (UF1), and their average as the scoring criteria. Evaluated using the established standards in the field and compared with previous work, our approach is shown to exhibit state-of-the-art performance.
Original languageEnglish
Title of host publicationFME'21: Proceedings of the 1st Workshop on Facial Micro-expression
Subtitle of host publicationAdvanced Techniques for Facial Expressions Generation and Spotting
Place of PublicationNew York
PublisherACM
Pages11-18
Number of pages8
ISBN (Print)9781450386838
DOIs
Publication statusPublished - 24 Oct 2021
EventFacial Micro-Expression (FME) Workshop and Challenge 2021: Advanced techniques for Facial Expressions Generation and Spotting - Chengdu, China
Duration: 20 Oct 202124 Oct 2021
https://megc2021.github.io/

Workshop

WorkshopFacial Micro-Expression (FME) Workshop and Challenge 2021
Abbreviated titleFME '21
Country/TerritoryChina
CityChengdu
Period20/10/2124/10/21
Internet address

Keywords

  • Facial Action Unit detection
  • Micro-expression analysis
  • Micro-movements

Fingerprint

Dive into the research topics of 'Facial action unit detection with local key facial sub-region based multi-label classification for micro-expression analysis'. Together they form a unique fingerprint.

Cite this