Higher Order Polynomial Transformer for Fine-Grained Freezing of Gait Detection

Renfei Sun, Kun Hu, Kaylena A.Ehgoetz Martens, Markus Hagenbuchner, Ah Chung Tsoi, Mohammed Bennamoun, Simon J.G. Lewis, Zhiyong Wang

Research output: Contribution to journalArticlepeer-review

5 Citations (Scopus)


Freezing of Gait (FoG) is a common symptom of Parkinson’s disease (PD), manifesting as a brief, episodic absence, or marked reduction in walking, despite a patient’s intention to move. Clinical assessment of FoG events from manual observations by experts is both time-consuming and highly subjective. Therefore, machine learning-based FoG identification methods would be desirable. In this article, we address this task as a fine-grained human action recognition problem based on vision inputs. A novel deep learning architecture, namely, higher order polynomial transformer (HP-Transformer), is proposed to incorporate pose and appearance feature sequences to formulate fine-grained FoG patterns. In particular, a higher order self-attention mechanism is proposed based on higher order polynomials. To this end, linear, bilinear, and trilinear transformers are formulated in pursuit of discriminative fine-grained representations. These representations are treated as multiple streams and further fused by a cross-order fusion strategy for FoG detection. Comprehensive experiments on a large in-house dataset collected during clinical assessments demonstrate the effectiveness of the proposed method, and an area under the receiver operating characteristic (ROC) curve (AUC) of 0.92 is achieved for detecting FoG.

Original languageEnglish
Pages (from-to)1-14
Number of pages14
JournalIEEE Transactions on Neural Networks and Learning Systems
Publication statusE-pub ahead of print - 12 Apr 2023


Dive into the research topics of 'Higher Order Polynomial Transformer for Fine-Grained Freezing of Gait Detection'. Together they form a unique fingerprint.

Cite this