Contrastive Self-Supervised Learning Leads to Higher Adversarial Susceptibility

Rohit Gupta, Naveed Akhtar, Ajmal Mian, Mubarak Shah

Research output: Chapter in Book/Conference paperConference paperpeer-review


Contrastive self-supervised learning (CSL) has managed to match or surpass the performance of supervised learning in image and video classification. However, it is still largely unknown if the nature of the representations induced by the two learning paradigms is similar. We investigate this under the lens of adversarial robustness. Our analysis of the problem reveals that CSL has intrinsically higher sensitivity to perturbations over supervised learning. We identify the uniform distribution of data representation over a unit hypersphere in the CSL representation space as the key contributor to this phenomenon. We establish that this is a result of the presence of false negative pairs in the training process, which increases model sensitivity to input perturbations. Our finding is supported by extensive experiments for image and video classification using adversarial perturbations and other input corruptions. We devise a strategy to detect and remove false negative pairs that is simple, yet effective in improving model robustness with CSL training. We close up to 68% of the robustness gap between CSL and its supervised counterpart. Finally, we contribute to adversarial learning by incorporating our method in CSL. We demonstrate an average gain of about 5% over two different state-of-the-art methods in this domain.

Original languageEnglish
Title of host publicationProceedings of the 37th AAAI Conference on Artificial Intelligence (AAAI 2023)
EditorsBrian Williams, Yiling Chen, Jennifer Neville
PublisherAAAI Press
Number of pages9
ISBN (Electronic)9781577358800
Publication statusPublished - 27 Jun 2023
Event37th AAAI Conference on Artificial Intelligence - Washington, United States
Duration: 7 Feb 202314 Feb 2023

Publication series

NameProceedings of the 37th AAAI Conference on Artificial Intelligence, AAAI 2023


Conference37th AAAI Conference on Artificial Intelligence
Abbreviated titleAAAI 2023
Country/TerritoryUnited States

Cite this