Global regularizer and temporal-aware cross-entropy for skeleton-based early action recognition

Research output: Chapter in Book/Conference paperConference paper

Abstract

In this paper, we propose a new approach to recognize the class label of an action before this action is fully performed based on skeleton sequences. Compared to action recognition which uses fully observed action sequences, early action recognition with partial sequences is much more challenging mainly due to: (1) the global information of a long-term action is not available in the partial sequence, and (2) the partial sequences at different observation ratios of an action contain a number of sub-actions with diverse motion information. To address the first challenge, we introduce a global regularizer to learn a hidden feature space, where the statistical properties of the partial sequences are similar to those of the full sequences. We introduce a temporal-aware cross-entropy to address the second challenge and achieve better prediction performance. We evaluate the proposed method on three challenging skeleton datasets. Experimental results show the superiority of the proposed method for skeleton-based early action recognition.

Original languageEnglish
Title of host publicationComputer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers
EditorsGreg Mori, Hongdong Li, Konrad Schindler, C.V. Jawahar
PublisherSpringer-Verlag Berlin
Pages729-745
Number of pages17
ISBN (Print)9783030208691
DOIs
Publication statusPublished - 2019
Event14th Asian Conference on Computer Vision, ACCV 2018 - Perth, Australia
Duration: 2 Dec 20186 Dec 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11364 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference14th Asian Conference on Computer Vision, ACCV 2018
CountryAustralia
CityPerth
Period2/12/186/12/18

Fingerprint

Cross-entropy
Action Recognition
Skeleton
Labels
Entropy
Partial
Performance Prediction
Feature Space
Statistical property
Motion
Evaluate
Experimental Results

Cite this

Ke, Q., Liu, J., Bennamoun, M., Rahmani, H., An, S., Sohel, F., & Boussaid, F. (2019). Global regularizer and temporal-aware cross-entropy for skeleton-based early action recognition. In G. Mori, H. Li, K. Schindler, & C. V. Jawahar (Eds.), Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers (pp. 729-745). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11364 LNCS). Springer-Verlag Berlin. https://doi.org/10.1007/978-3-030-20870-7_45
Ke, Qiuhong ; Liu, Jun ; Bennamoun, Mohammed ; Rahmani, Hossein ; An, Senjian ; Sohel, Ferdous ; Boussaid, Farid. / Global regularizer and temporal-aware cross-entropy for skeleton-based early action recognition. Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers. editor / Greg Mori ; Hongdong Li ; Konrad Schindler ; C.V. Jawahar. Springer-Verlag Berlin, 2019. pp. 729-745 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{bdc4da884e484b4c8267a31faed37b6d,
title = "Global regularizer and temporal-aware cross-entropy for skeleton-based early action recognition",
abstract = "In this paper, we propose a new approach to recognize the class label of an action before this action is fully performed based on skeleton sequences. Compared to action recognition which uses fully observed action sequences, early action recognition with partial sequences is much more challenging mainly due to: (1) the global information of a long-term action is not available in the partial sequence, and (2) the partial sequences at different observation ratios of an action contain a number of sub-actions with diverse motion information. To address the first challenge, we introduce a global regularizer to learn a hidden feature space, where the statistical properties of the partial sequences are similar to those of the full sequences. We introduce a temporal-aware cross-entropy to address the second challenge and achieve better prediction performance. We evaluate the proposed method on three challenging skeleton datasets. Experimental results show the superiority of the proposed method for skeleton-based early action recognition.",
keywords = "3D skeleton sequences, Early action recognition, Global regularizer, Temporal-aware cross-entropy",
author = "Qiuhong Ke and Jun Liu and Mohammed Bennamoun and Hossein Rahmani and Senjian An and Ferdous Sohel and Farid Boussaid",
year = "2019",
doi = "10.1007/978-3-030-20870-7_45",
language = "English",
isbn = "9783030208691",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer-Verlag Berlin",
pages = "729--745",
editor = "Greg Mori and Hongdong Li and Konrad Schindler and C.V. Jawahar",
booktitle = "Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers",

}

Ke, Q, Liu, J, Bennamoun, M, Rahmani, H, An, S, Sohel, F & Boussaid, F 2019, Global regularizer and temporal-aware cross-entropy for skeleton-based early action recognition. in G Mori, H Li, K Schindler & CV Jawahar (eds), Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11364 LNCS, Springer-Verlag Berlin, pp. 729-745, 14th Asian Conference on Computer Vision, ACCV 2018, Perth, Australia, 2/12/18. https://doi.org/10.1007/978-3-030-20870-7_45

Global regularizer and temporal-aware cross-entropy for skeleton-based early action recognition. / Ke, Qiuhong; Liu, Jun; Bennamoun, Mohammed; Rahmani, Hossein; An, Senjian; Sohel, Ferdous; Boussaid, Farid.

Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers. ed. / Greg Mori; Hongdong Li; Konrad Schindler; C.V. Jawahar. Springer-Verlag Berlin, 2019. p. 729-745 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11364 LNCS).

Research output: Chapter in Book/Conference paperConference paper

TY - GEN

T1 - Global regularizer and temporal-aware cross-entropy for skeleton-based early action recognition

AU - Ke, Qiuhong

AU - Liu, Jun

AU - Bennamoun, Mohammed

AU - Rahmani, Hossein

AU - An, Senjian

AU - Sohel, Ferdous

AU - Boussaid, Farid

PY - 2019

Y1 - 2019

N2 - In this paper, we propose a new approach to recognize the class label of an action before this action is fully performed based on skeleton sequences. Compared to action recognition which uses fully observed action sequences, early action recognition with partial sequences is much more challenging mainly due to: (1) the global information of a long-term action is not available in the partial sequence, and (2) the partial sequences at different observation ratios of an action contain a number of sub-actions with diverse motion information. To address the first challenge, we introduce a global regularizer to learn a hidden feature space, where the statistical properties of the partial sequences are similar to those of the full sequences. We introduce a temporal-aware cross-entropy to address the second challenge and achieve better prediction performance. We evaluate the proposed method on three challenging skeleton datasets. Experimental results show the superiority of the proposed method for skeleton-based early action recognition.

AB - In this paper, we propose a new approach to recognize the class label of an action before this action is fully performed based on skeleton sequences. Compared to action recognition which uses fully observed action sequences, early action recognition with partial sequences is much more challenging mainly due to: (1) the global information of a long-term action is not available in the partial sequence, and (2) the partial sequences at different observation ratios of an action contain a number of sub-actions with diverse motion information. To address the first challenge, we introduce a global regularizer to learn a hidden feature space, where the statistical properties of the partial sequences are similar to those of the full sequences. We introduce a temporal-aware cross-entropy to address the second challenge and achieve better prediction performance. We evaluate the proposed method on three challenging skeleton datasets. Experimental results show the superiority of the proposed method for skeleton-based early action recognition.

KW - 3D skeleton sequences

KW - Early action recognition

KW - Global regularizer

KW - Temporal-aware cross-entropy

UR - http://www.scopus.com/inward/record.url?scp=85066875250&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-20870-7_45

DO - 10.1007/978-3-030-20870-7_45

M3 - Conference paper

SN - 9783030208691

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 729

EP - 745

BT - Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers

A2 - Mori, Greg

A2 - Li, Hongdong

A2 - Schindler, Konrad

A2 - Jawahar, C.V.

PB - Springer-Verlag Berlin

ER -

Ke Q, Liu J, Bennamoun M, Rahmani H, An S, Sohel F et al. Global regularizer and temporal-aware cross-entropy for skeleton-based early action recognition. In Mori G, Li H, Schindler K, Jawahar CV, editors, Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers. Springer-Verlag Berlin. 2019. p. 729-745. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-030-20870-7_45