Towards a fully automatic markerless motion analysis system for the estimation of body joint kinematics with application to sport analysis

Copyright © 2015 SCITEPRESS - Science and Technology Publications. All rights reserved. We present a complete markerless motion analysis system for the estimation of human body joint kinematics (JK) in sports. The system works in two phases, an offline phase in which a reference 3D scan (model) of the athlete is acquired. A skeleton consisting of a number of articulated joints is then fitted to the model based on anatomical data. In the online phase, a number of video cameras are utilized to capture the athlete motion from different viewpoints. In order to reconstruct the 3D motion of the athlete, a method is developed to estimate the intrinsic and the extrinsic parameters of all cameras. Synchronized videos are then captured of the athlete while performing an action then a kernel density estimator based background segmentation algorithm is applied to extract the silhouettes of the athlete in each video which are then reprojected to their 3D source to reconstruct an estimate of the body shape (visual hull). To estimate the JK, constrained optimization techniques are then used to extract information from the shape and the skeleton of the reconstructed VH. This includes a number of geometrically-driven cost functions driven by mathematical and physical constraints of the human body to align and deform the reference 3D model to the shape of the VH. Unlike existing pose estimation approaches where a reference pose with labeled body parts are used to initialize the pose estimation process, our approach identifies the the body shape and parts automatically and without any assumptions of its temporal continuity when dealing with unrelated frames which is a very challenging task due to the high dimensionality of the pose space. The proposed system modules are tested with real examples and achieved a promising results.