3DMODT: Attention-Guided Affinities for Joint Detection & Tracking in 3D Point Clouds

Jyoti Kini, Ajmal Mian, Mubarak Shah

Research output: Chapter in Book/Conference paperConference paperpeer-review

6 Citations (Scopus)

Abstract

We propose a method for joint detection and tracking of multiple objects in 3D point clouds, a task conventionally treated as a two-step process comprising object detection followed by data association. Our method embeds both steps into a single end-to-end trainable network eliminating the dependency on external object detectors. Our model exploits temporal information employing multiple frames to detect objects and track them in a single network, thereby making it a utilitarian formulation for real-world scenarios. Computing affinity matrix by employing features similarity across consecutive point cloud scans forms an integral part of visual tracking. We propose an attention-based refinement module to refine the affinity matrix by suppressing erroneous correspondences. The module is designed to capture the global context in affinity matrix by employing self-attention within each affinity matrix and cross-attention across a pair of affinity matrices. Unlike competing approaches, our network does not require complex post-processing algorithms, and directly processes raw LiDAR frames to output tracking results. We demonstrate the effectiveness of our method on three tracking benchmarks: JRDB, Waymo, and KITTI. Experimental evaluations indicate the ability of our model to generalize well across datasets.

Original languageEnglish
Title of host publicationProceedings - ICRA 2023
Subtitle of host publicationIEEE International Conference on Robotics and Automation
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages841-848
Number of pages8
ISBN (Electronic)9798350323658
DOIs
Publication statusPublished - 2023
Event2023 IEEE International Conference on Robotics and Automation - London, United Kingdom
Duration: 29 May 20232 Jun 2023

Publication series

NameProceedings - IEEE International Conference on Robotics and Automation
Volume2023-May
ISSN (Print)1050-4729

Conference

Conference2023 IEEE International Conference on Robotics and Automation
Abbreviated titleICRA 2023
Country/TerritoryUnited Kingdom
CityLondon
Period29/05/232/06/23

Fingerprint

Dive into the research topics of '3DMODT: Attention-Guided Affinities for Joint Detection & Tracking in 3D Point Clouds'. Together they form a unique fingerprint.

Cite this