Global regularizer and temporal-aware cross-entropy for skeleton-based early action recognition

Qiuhong Ke; Jun Liu; Mohammed Bennamoun; Hossein Rahmani; Senjian An; Ferdous Sohel; Farid Boussaid

doi:10.1007/978-3-030-20870-7_45

Global regularizer and temporal-aware cross-entropy for skeleton-based early action recognition

Qiuhong Ke, Jun Liu, Mohammed Bennamoun, Hossein Rahmani, Senjian An, Ferdous Sohel, Farid Boussaid

Research output: Chapter in Book/Conference paper › Conference paper › peer-review

3 Citations (Scopus)

Abstract

In this paper, we propose a new approach to recognize the class label of an action before this action is fully performed based on skeleton sequences. Compared to action recognition which uses fully observed action sequences, early action recognition with partial sequences is much more challenging mainly due to: (1) the global information of a long-term action is not available in the partial sequence, and (2) the partial sequences at different observation ratios of an action contain a number of sub-actions with diverse motion information. To address the first challenge, we introduce a global regularizer to learn a hidden feature space, where the statistical properties of the partial sequences are similar to those of the full sequences. We introduce a temporal-aware cross-entropy to address the second challenge and achieve better prediction performance. We evaluate the proposed method on three challenging skeleton datasets. Experimental results show the superiority of the proposed method for skeleton-based early action recognition.

Original language	English
Title of host publication	Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers
Editors	C.V. Jawahar, Hongdong Li, Greg Mori, Konrad Schindler
Publisher	Springer-Verlag Berlin
Pages	729-745
Number of pages	17
ISBN (Print)	9783030208691
DOIs	https://doi.org/10.1007/978-3-030-20870-7_45
Publication status	Published - 2019
Event	14th Asian Conference on Computer Vision, ACCV 2018 - Perth, Australia Duration: 2 Dec 2018 → 6 Dec 2018

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11364 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	14th Asian Conference on Computer Vision, ACCV 2018
Country/Territory	Australia
City	Perth
Period	2/12/18 → 6/12/18

Access to Document

10.1007/978-3-030-20870-7_45

https://eprints.lancs.ac.uk/id/eprint/128786/1/0050.pdf

Cite this

Ke, Q., Liu, J., Bennamoun, M., Rahmani, H., An, S., Sohel, F., & Boussaid, F. (2019). Global regularizer and temporal-aware cross-entropy for skeleton-based early action recognition. In C. V. Jawahar, H. Li, G. Mori, & K. Schindler (Eds.), Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers (pp. 729-745). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11364 LNCS). Springer-Verlag Berlin. https://doi.org/10.1007/978-3-030-20870-7_45

Ke, Qiuhong ; Liu, Jun ; Bennamoun, Mohammed et al. / Global regularizer and temporal-aware cross-entropy for skeleton-based early action recognition. Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers. editor / C.V. Jawahar ; Hongdong Li ; Greg Mori ; Konrad Schindler. Springer-Verlag Berlin, 2019. pp. 729-745 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{bdc4da884e484b4c8267a31faed37b6d,

title = "Global regularizer and temporal-aware cross-entropy for skeleton-based early action recognition",

abstract = "In this paper, we propose a new approach to recognize the class label of an action before this action is fully performed based on skeleton sequences. Compared to action recognition which uses fully observed action sequences, early action recognition with partial sequences is much more challenging mainly due to: (1) the global information of a long-term action is not available in the partial sequence, and (2) the partial sequences at different observation ratios of an action contain a number of sub-actions with diverse motion information. To address the first challenge, we introduce a global regularizer to learn a hidden feature space, where the statistical properties of the partial sequences are similar to those of the full sequences. We introduce a temporal-aware cross-entropy to address the second challenge and achieve better prediction performance. We evaluate the proposed method on three challenging skeleton datasets. Experimental results show the superiority of the proposed method for skeleton-based early action recognition.",

keywords = "3D skeleton sequences, Early action recognition, Global regularizer, Temporal-aware cross-entropy",

author = "Qiuhong Ke and Jun Liu and Mohammed Bennamoun and Hossein Rahmani and Senjian An and Ferdous Sohel and Farid Boussaid",

year = "2019",

doi = "10.1007/978-3-030-20870-7_45",

language = "English",

isbn = "9783030208691",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer-Verlag Berlin",

pages = "729--745",

editor = "C.V. Jawahar and Hongdong Li and Greg Mori and Konrad Schindler",

booktitle = "Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers",

address = "Germany",

note = "14th Asian Conference on Computer Vision, ACCV 2018 ; Conference date: 02-12-2018 Through 06-12-2018",

}

Ke, Q, Liu, J, Bennamoun, M, Rahmani, H, An, S, Sohel, F & Boussaid, F 2019, Global regularizer and temporal-aware cross-entropy for skeleton-based early action recognition. in CV Jawahar, H Li, G Mori & K Schindler (eds), Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11364 LNCS, Springer-Verlag Berlin, pp. 729-745, 14th Asian Conference on Computer Vision, ACCV 2018, Perth, Western Australia, Australia, 2/12/18. https://doi.org/10.1007/978-3-030-20870-7_45

Global regularizer and temporal-aware cross-entropy for skeleton-based early action recognition. / Ke, Qiuhong; Liu, Jun; Bennamoun, Mohammed et al.
Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers. ed. / C.V. Jawahar; Hongdong Li; Greg Mori; Konrad Schindler. Springer-Verlag Berlin, 2019. p. 729-745 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11364 LNCS).

Research output: Chapter in Book/Conference paper › Conference paper › peer-review

TY - GEN

T1 - Global regularizer and temporal-aware cross-entropy for skeleton-based early action recognition

AU - Ke, Qiuhong

AU - Liu, Jun

AU - Bennamoun, Mohammed

AU - Rahmani, Hossein

AU - An, Senjian

AU - Sohel, Ferdous

AU - Boussaid, Farid

PY - 2019

Y1 - 2019

N2 - In this paper, we propose a new approach to recognize the class label of an action before this action is fully performed based on skeleton sequences. Compared to action recognition which uses fully observed action sequences, early action recognition with partial sequences is much more challenging mainly due to: (1) the global information of a long-term action is not available in the partial sequence, and (2) the partial sequences at different observation ratios of an action contain a number of sub-actions with diverse motion information. To address the first challenge, we introduce a global regularizer to learn a hidden feature space, where the statistical properties of the partial sequences are similar to those of the full sequences. We introduce a temporal-aware cross-entropy to address the second challenge and achieve better prediction performance. We evaluate the proposed method on three challenging skeleton datasets. Experimental results show the superiority of the proposed method for skeleton-based early action recognition.

AB - In this paper, we propose a new approach to recognize the class label of an action before this action is fully performed based on skeleton sequences. Compared to action recognition which uses fully observed action sequences, early action recognition with partial sequences is much more challenging mainly due to: (1) the global information of a long-term action is not available in the partial sequence, and (2) the partial sequences at different observation ratios of an action contain a number of sub-actions with diverse motion information. To address the first challenge, we introduce a global regularizer to learn a hidden feature space, where the statistical properties of the partial sequences are similar to those of the full sequences. We introduce a temporal-aware cross-entropy to address the second challenge and achieve better prediction performance. We evaluate the proposed method on three challenging skeleton datasets. Experimental results show the superiority of the proposed method for skeleton-based early action recognition.

KW - 3D skeleton sequences

KW - Early action recognition

KW - Global regularizer

KW - Temporal-aware cross-entropy

UR - http://www.scopus.com/inward/record.url?scp=85066875250&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-20870-7_45

DO - 10.1007/978-3-030-20870-7_45

M3 - Conference paper

AN - SCOPUS:85066875250

SN - 9783030208691

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 729

EP - 745

BT - Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers

A2 - Jawahar, C.V.

A2 - Li, Hongdong

A2 - Mori, Greg

A2 - Schindler, Konrad

PB - Springer-Verlag Berlin

T2 - 14th Asian Conference on Computer Vision, ACCV 2018

Y2 - 2 December 2018 through 6 December 2018

ER -

Ke Q, Liu J, Bennamoun M, Rahmani H, An S, Sohel F et al. Global regularizer and temporal-aware cross-entropy for skeleton-based early action recognition. In Jawahar CV, Li H, Mori G, Schindler K, editors, Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers. Springer-Verlag Berlin. 2019. p. 729-745. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-20870-7_45

Global regularizer and temporal-aware cross-entropy for skeleton-based early action recognition

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Advanced Computer Vision Techniques for Marine Ecology

Advanced 3D Computer Vision Algorithms for 'Find and Grasp' Future Robots

Cite this

Global regularizer and temporal-aware cross-entropy for skeleton-based early action recognition

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Projects

Advanced Computer Vision Techniques for Marine Ecology

Advanced 3D Computer Vision Algorithms for 'Find and Grasp' Future Robots

Cite this