Domain-Generalized Robotic Picking via Contrastive Learning-Based 6-D Pose Estimation

Jian Liu, Wei Sun, Hui Yang, Chongpei Liu, Xing Zhang, Ajmal Mian

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)


Vision-guided robotic picking in 3-D space is a key technology for industrial automation and intelligent manufacturing. However, existing methods rely on labeled real-world data for learning, significantly limiting their ability to generalize to novel objects and robustness to challenging scenes containing occlusions and clutter. To address these problems, we propose a domain-generalized robotic picking method (DGPF6D) that builds on contrastive learning-based 6-D pose estimation. DGPF6D generalizes to real-world scenes by training only on synthetic data and without using shape priors. Specifically, we first perform continuous data augmentations on the synthetic RGB and point cloud images such that they can better simulate real-world scenes with occlusions and clutter. We then feed the augmented images in parallel to a two-stage (<italic/>i.e.<italic/>, 3-D shape reconstruction and 6-D pose estimation) contrastive learning framework, thereby enhancing the domain-generalization ability and robustness of DGPF6D. Moreover, we propose a point cloud cross attention-guided intracategory unknown object 3-D shape reconstruction network, which can effectively fuse the observed and the unit random point clouds and explicitly highlight their differences, thus avoiding the dependence of DGPF6D on shape priors. Finally, we build a robotic picking system employing DGPF6D to realize domain-generalized robotic picking in 3-D space. Extensive experiments on two benchmarks and real-world scenes show that DGPF6D achieves state-of-the-art performance, and can be effectively applied for domain-generalized robotic picking.

Original languageEnglish
Pages (from-to)8650-8661
Number of pages12
JournalIEEE Transactions on Industrial Informatics
Issue number6
Early online date20 Mar 2024
Publication statusPublished - 1 Jun 2024


Dive into the research topics of 'Domain-Generalized Robotic Picking via Contrastive Learning-Based 6-D Pose Estimation'. Together they form a unique fingerprint.

Cite this