Learning interpretable expression-sensitive features for 3D dynamic facial expression recognition

Mingliang Xue, Ajmal Mian, Xiaodong Duan, Wanquan Liu

Research output: Chapter in Book/Conference paperConference paper

Abstract

Different facial components carry different amount of information being conveyed for 3D dynamic expression recognition. Hence, identifying facial components that are highly relevant to specific expression changes is crucial for discriminative facial expression recognition. This work aims to learn expression-sensitive features, which are expected to not only yield comparable recognition performance with the state-of-the-art ones, but also can be interpreted by human. Firstly, spatio-temporal features (HOG3D) are extracted from local depth patch-sequences to represent facial expression dynamics. A two-phase feature selection process is then proposed to determine the facial components that can best distinguish the expressions. In order to verify the effectiveness of the resulting facial components, the expression-sensitive features from the corresponding area are fed into a hierarchical classifier for facial expression recognition. The proposed method is evaluated on the BU-4DFE benchmark database, and results show that learned expression-sensitive features can achieve a comparable recognition performance with existing methods. Additionally, the resulting HOG3D features after feature selection can be used to generate semantic interpretation of the expression dynamics.

Original languageEnglish
Title of host publicationProceedings - 14th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2019
PublisherIEEE, Institute of Electrical and Electronics Engineers
ISBN (Electronic)9781728100890
DOIs
Publication statusPublished - 2019
Event14th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2019 - Lille, France
Duration: 14 May 201918 May 2019

Publication series

NameProceedings - 14th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2019

Conference

Conference14th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2019
CountryFrance
CityLille
Period14/05/1918/05/19

Fingerprint

Feature extraction
Classifiers
Semantics

Cite this

Xue, M., Mian, A., Duan, X., & Liu, W. (2019). Learning interpretable expression-sensitive features for 3D dynamic facial expression recognition. In Proceedings - 14th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2019 [8756564] (Proceedings - 14th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2019). IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/FG.2019.8756564
Xue, Mingliang ; Mian, Ajmal ; Duan, Xiaodong ; Liu, Wanquan. / Learning interpretable expression-sensitive features for 3D dynamic facial expression recognition. Proceedings - 14th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2019. IEEE, Institute of Electrical and Electronics Engineers, 2019. (Proceedings - 14th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2019).
@inproceedings{a47fe17b9a6e4756827cb6c02cae7b34,
title = "Learning interpretable expression-sensitive features for 3D dynamic facial expression recognition",
abstract = "Different facial components carry different amount of information being conveyed for 3D dynamic expression recognition. Hence, identifying facial components that are highly relevant to specific expression changes is crucial for discriminative facial expression recognition. This work aims to learn expression-sensitive features, which are expected to not only yield comparable recognition performance with the state-of-the-art ones, but also can be interpreted by human. Firstly, spatio-temporal features (HOG3D) are extracted from local depth patch-sequences to represent facial expression dynamics. A two-phase feature selection process is then proposed to determine the facial components that can best distinguish the expressions. In order to verify the effectiveness of the resulting facial components, the expression-sensitive features from the corresponding area are fed into a hierarchical classifier for facial expression recognition. The proposed method is evaluated on the BU-4DFE benchmark database, and results show that learned expression-sensitive features can achieve a comparable recognition performance with existing methods. Additionally, the resulting HOG3D features after feature selection can be used to generate semantic interpretation of the expression dynamics.",
author = "Mingliang Xue and Ajmal Mian and Xiaodong Duan and Wanquan Liu",
year = "2019",
doi = "10.1109/FG.2019.8756564",
language = "English",
series = "Proceedings - 14th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2019",
publisher = "IEEE, Institute of Electrical and Electronics Engineers",
booktitle = "Proceedings - 14th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2019",
address = "United States",

}

Xue, M, Mian, A, Duan, X & Liu, W 2019, Learning interpretable expression-sensitive features for 3D dynamic facial expression recognition. in Proceedings - 14th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2019., 8756564, Proceedings - 14th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2019, IEEE, Institute of Electrical and Electronics Engineers, 14th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2019, Lille, France, 14/05/19. https://doi.org/10.1109/FG.2019.8756564

Learning interpretable expression-sensitive features for 3D dynamic facial expression recognition. / Xue, Mingliang; Mian, Ajmal; Duan, Xiaodong; Liu, Wanquan.

Proceedings - 14th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2019. IEEE, Institute of Electrical and Electronics Engineers, 2019. 8756564 (Proceedings - 14th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2019).

Research output: Chapter in Book/Conference paperConference paper

TY - GEN

T1 - Learning interpretable expression-sensitive features for 3D dynamic facial expression recognition

AU - Xue, Mingliang

AU - Mian, Ajmal

AU - Duan, Xiaodong

AU - Liu, Wanquan

PY - 2019

Y1 - 2019

N2 - Different facial components carry different amount of information being conveyed for 3D dynamic expression recognition. Hence, identifying facial components that are highly relevant to specific expression changes is crucial for discriminative facial expression recognition. This work aims to learn expression-sensitive features, which are expected to not only yield comparable recognition performance with the state-of-the-art ones, but also can be interpreted by human. Firstly, spatio-temporal features (HOG3D) are extracted from local depth patch-sequences to represent facial expression dynamics. A two-phase feature selection process is then proposed to determine the facial components that can best distinguish the expressions. In order to verify the effectiveness of the resulting facial components, the expression-sensitive features from the corresponding area are fed into a hierarchical classifier for facial expression recognition. The proposed method is evaluated on the BU-4DFE benchmark database, and results show that learned expression-sensitive features can achieve a comparable recognition performance with existing methods. Additionally, the resulting HOG3D features after feature selection can be used to generate semantic interpretation of the expression dynamics.

AB - Different facial components carry different amount of information being conveyed for 3D dynamic expression recognition. Hence, identifying facial components that are highly relevant to specific expression changes is crucial for discriminative facial expression recognition. This work aims to learn expression-sensitive features, which are expected to not only yield comparable recognition performance with the state-of-the-art ones, but also can be interpreted by human. Firstly, spatio-temporal features (HOG3D) are extracted from local depth patch-sequences to represent facial expression dynamics. A two-phase feature selection process is then proposed to determine the facial components that can best distinguish the expressions. In order to verify the effectiveness of the resulting facial components, the expression-sensitive features from the corresponding area are fed into a hierarchical classifier for facial expression recognition. The proposed method is evaluated on the BU-4DFE benchmark database, and results show that learned expression-sensitive features can achieve a comparable recognition performance with existing methods. Additionally, the resulting HOG3D features after feature selection can be used to generate semantic interpretation of the expression dynamics.

UR - http://www.scopus.com/inward/record.url?scp=85070441958&partnerID=8YFLogxK

U2 - 10.1109/FG.2019.8756564

DO - 10.1109/FG.2019.8756564

M3 - Conference paper

T3 - Proceedings - 14th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2019

BT - Proceedings - 14th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2019

PB - IEEE, Institute of Electrical and Electronics Engineers

ER -

Xue M, Mian A, Duan X, Liu W. Learning interpretable expression-sensitive features for 3D dynamic facial expression recognition. In Proceedings - 14th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2019. IEEE, Institute of Electrical and Electronics Engineers. 2019. 8756564. (Proceedings - 14th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2019). https://doi.org/10.1109/FG.2019.8756564