WEmbSim: A Simple yet Effective Metric for Image Captioning

Naeha Sharif, Lyndon White, Mohammed Bennamoun, Wei Liu, Syed Afaq Ali Shah

Research output: Chapter in Book/Conference paperConference paper

Abstract

The area of automatic image caption evaluation is still undergoing intensive research to address the needs of generating captions which can meet adequacy and fluency requirements. Based on our past attempts at developing highly sophisticated learning-based metrics, we have discovered that a simple cosine similarity measure using the Mean of Word Embeddings (MOWE) of captions can actually achieve a surprisingly high performance on unsupervised caption evaluation. This inspires our proposed work on an effective metric WEmbSim, which beats complex measures such as SPICE, CIDEr and WMD at system-level correlation with human judgments. Moreover, it also
achieves the best accuracy at matching human consensus scores for caption pairs, against commonly used unsupervised methods. Therefore, we believe that WEmbSim sets a new baseline for any complex metric to be justified.
Original languageEnglish
Title of host publicationDigital Image Computing: Techniques and Applications, 2020 (DICTA 2020)
Place of PublicationUSA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Publication statusPublished - 30 Nov 2020
EventDigital Image Computing: Technqiues and Applications 2020 - , Virtual
Duration: 30 Nov 20202 Dec 2020

Conference

ConferenceDigital Image Computing: Technqiues and Applications 2020
Abbreviated titleDICTA2020
CountryVirtual
Period30/11/202/12/20

Fingerprint Dive into the research topics of 'WEmbSim: A Simple yet Effective Metric for Image Captioning'. Together they form a unique fingerprint.

Cite this