A new representation of skeleton sequences for 3D action recognition

Research output: Chapter in Book/Conference paperConference paper

86 Citations (Scopus)

Abstract

This paper presents a new method for 3D action recognition
with skeleton sequences (i.e., 3D trajectories of human
skeleton joints). The proposed method first transforms each
skeleton sequence into three clips each consisting of several
frames for spatial temporal feature learning using deep
neural networks. Each clip is generated from one channel
of the cylindrical coordinates of the skeleton sequence.
Each frame of the generated clips represents the temporal
information of the entire skeleton sequence, and incorporates
one particular spatial relationship between the joints.
The entire clips include multiple frames with different spatial
relationships, which provide useful spatial structural information
of the human skeleton. We propose to use deep
convolutional neural networks to learn long-term temporal
information of the skeleton sequence from the frames of the
generated clips, and then use a Multi-Task Learning Network
(MTLN) to jointly process all frames of the generated
clips in parallel to incorporate spatial structural information
for action recognition. Experimental results clearly
show the effectiveness of the proposed new representation
and feature learning method for 3D action recognition.
Original languageEnglish
Title of host publicationProceedings
Subtitle of host publication30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017
Place of PublicationUnited States
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages4570-4579
ISBN (Print)9781538604571
Publication statusPublished - 2017
Event30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017 - Honolulu, United States
Duration: 21 Jul 201726 Jul 2017

Conference

Conference30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017
CountryUnited States
CityHonolulu
Period21/07/1726/07/17

Fingerprint

Trajectories
Neural networks

Cite this

Ke, Q., Bennamoun, M., An, S., Sohel, F., & Boussaid, F. (2017). A new representation of skeleton sequences for 3D action recognition. In Proceedings: 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017 (pp. 4570-4579). United States: IEEE, Institute of Electrical and Electronics Engineers.
Ke, Qiuhong ; Bennamoun, Mohammed ; An, Senjian ; Sohel, Ferdous ; Boussaid, Farid. / A new representation of skeleton sequences for 3D action recognition. Proceedings: 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017. United States : IEEE, Institute of Electrical and Electronics Engineers, 2017. pp. 4570-4579
@inproceedings{d80aab86e7a14d1590e218cfdd4f3a15,
title = "A new representation of skeleton sequences for 3D action recognition",
abstract = "This paper presents a new method for 3D action recognitionwith skeleton sequences (i.e., 3D trajectories of humanskeleton joints). The proposed method first transforms eachskeleton sequence into three clips each consisting of severalframes for spatial temporal feature learning using deepneural networks. Each clip is generated from one channelof the cylindrical coordinates of the skeleton sequence.Each frame of the generated clips represents the temporalinformation of the entire skeleton sequence, and incorporatesone particular spatial relationship between the joints.The entire clips include multiple frames with different spatialrelationships, which provide useful spatial structural informationof the human skeleton. We propose to use deepconvolutional neural networks to learn long-term temporalinformation of the skeleton sequence from the frames of thegenerated clips, and then use a Multi-Task Learning Network(MTLN) to jointly process all frames of the generatedclips in parallel to incorporate spatial structural informationfor action recognition. Experimental results clearlyshow the effectiveness of the proposed new representationand feature learning method for 3D action recognition.",
author = "Qiuhong Ke and Mohammed Bennamoun and Senjian An and Ferdous Sohel and Farid Boussaid",
year = "2017",
language = "English",
isbn = "9781538604571",
pages = "4570--4579",
booktitle = "Proceedings",
publisher = "IEEE, Institute of Electrical and Electronics Engineers",
address = "United States",

}

Ke, Q, Bennamoun, M, An, S, Sohel, F & Boussaid, F 2017, A new representation of skeleton sequences for 3D action recognition. in Proceedings: 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017. IEEE, Institute of Electrical and Electronics Engineers, United States, pp. 4570-4579, 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, United States, 21/07/17.

A new representation of skeleton sequences for 3D action recognition. / Ke, Qiuhong; Bennamoun, Mohammed; An, Senjian; Sohel, Ferdous; Boussaid, Farid.

Proceedings: 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017. United States : IEEE, Institute of Electrical and Electronics Engineers, 2017. p. 4570-4579.

Research output: Chapter in Book/Conference paperConference paper

TY - GEN

T1 - A new representation of skeleton sequences for 3D action recognition

AU - Ke, Qiuhong

AU - Bennamoun, Mohammed

AU - An, Senjian

AU - Sohel, Ferdous

AU - Boussaid, Farid

PY - 2017

Y1 - 2017

N2 - This paper presents a new method for 3D action recognitionwith skeleton sequences (i.e., 3D trajectories of humanskeleton joints). The proposed method first transforms eachskeleton sequence into three clips each consisting of severalframes for spatial temporal feature learning using deepneural networks. Each clip is generated from one channelof the cylindrical coordinates of the skeleton sequence.Each frame of the generated clips represents the temporalinformation of the entire skeleton sequence, and incorporatesone particular spatial relationship between the joints.The entire clips include multiple frames with different spatialrelationships, which provide useful spatial structural informationof the human skeleton. We propose to use deepconvolutional neural networks to learn long-term temporalinformation of the skeleton sequence from the frames of thegenerated clips, and then use a Multi-Task Learning Network(MTLN) to jointly process all frames of the generatedclips in parallel to incorporate spatial structural informationfor action recognition. Experimental results clearlyshow the effectiveness of the proposed new representationand feature learning method for 3D action recognition.

AB - This paper presents a new method for 3D action recognitionwith skeleton sequences (i.e., 3D trajectories of humanskeleton joints). The proposed method first transforms eachskeleton sequence into three clips each consisting of severalframes for spatial temporal feature learning using deepneural networks. Each clip is generated from one channelof the cylindrical coordinates of the skeleton sequence.Each frame of the generated clips represents the temporalinformation of the entire skeleton sequence, and incorporatesone particular spatial relationship between the joints.The entire clips include multiple frames with different spatialrelationships, which provide useful spatial structural informationof the human skeleton. We propose to use deepconvolutional neural networks to learn long-term temporalinformation of the skeleton sequence from the frames of thegenerated clips, and then use a Multi-Task Learning Network(MTLN) to jointly process all frames of the generatedclips in parallel to incorporate spatial structural informationfor action recognition. Experimental results clearlyshow the effectiveness of the proposed new representationand feature learning method for 3D action recognition.

M3 - Conference paper

SN - 9781538604571

SP - 4570

EP - 4579

BT - Proceedings

PB - IEEE, Institute of Electrical and Electronics Engineers

CY - United States

ER -

Ke Q, Bennamoun M, An S, Sohel F, Boussaid F. A new representation of skeleton sequences for 3D action recognition. In Proceedings: 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017. United States: IEEE, Institute of Electrical and Electronics Engineers. 2017. p. 4570-4579