SubICap: Towards Subword-Informed Image Captioning

Research output: Chapter in Book/Conference paperConference paperpeer-review

2 Downloads (Pure)

Abstract

Existing Image Captioning (IC) systems model words as atomic units in captions and are unable to exploit the structural information in the words. This makes representation of rare words very difficult and out-of-vocabulary words impossible. Moreover, to avoid computational complexity, existing IC models operate over a modest sized vocabulary of frequent words, such that the identity of rare words is lost. In this work we address this common limitation of IC systems in dealing with rare words in the corpora. We decompose words into smaller constituent units ‘subwords’ and represent captions as a sequence of subwords instead of words. This helps represent all words in the corpora using a significantly lower subword vocabulary, leading to better parameter learning. Using subword language modeling, our captioning system improves various metric scores, with a training vocabulary size approximately 90% less than the baseline and various state-of-the-art word-level models. Our quantitative and qualitative results and analysis signify the efficacy of our proposed approach.
Original languageEnglish
Title of host publicationConference Proceedings 2021 IEEE Winter Conference on Applications of Computer Vision (WACV)
Place of PublicationUSA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages3540-3549
Number of pages10
ISBN (Electronic)978-1-6654-0477-8
DOIs
Publication statusPublished - 14 Jun 2021
Event2021 IEEE Winter Conference on Applications of Computer Vision - Virtual, Virtual
Duration: 5 Jan 20219 Jan 2021

Conference

Conference2021 IEEE Winter Conference on Applications of Computer Vision
Abbreviated titleWACV 2021
Country/TerritoryVirtual
Period5/01/219/01/21

Fingerprint

Dive into the research topics of 'SubICap: Towards Subword-Informed Image Captioning'. Together they form a unique fingerprint.

Cite this