Learning-based Composite Metrics for Improved Caption Evaluation

Naeha Sharif, Mohammed Bennamoun, Lyndon Rhys White, Syed Shah

Research output: Chapter in Book/Conference paperConference paper

2 Citations (Scopus)

Abstract

The evaluation of image caption quality is a challenging task, which requires the assessment of two main aspects in a caption: adequacy and fluency. These quality aspects can be judged using a combination of several linguistic features. However, most of the current image captioning metrics focus only on specific linguistic facets, such as the lexical or semantic, and fail to meet a satisfactory level of correlation with human judgements at the sentence-level. We propose a learning-based framework to incorporate the scores of a set of lexical and semantic metrics as features, to capture the adequacy and fluency of captions at different linguistic levels. Our experimental results demonstrate that composite metrics draw upon the strengths of standalone measures to yield improved correlation and accuracy.
Original languageEnglish
Title of host publicationProceedings of ACL 2018, Student Research Workshop
Place of PublicationAustralia
PublisherAssociation for Computational Linguistics
Pages14-20
Publication statusPublished - 2018
Event56th Annual Meeting of Association for Computational Linguistics - Melbourne, Australia
Duration: 15 Jul 201820 Jul 2018

Conference

Conference56th Annual Meeting of Association for Computational Linguistics
Abbreviated titleACL2018
CountryAustralia
CityMelbourne
Period15/07/1820/07/18

Fingerprint

Linguistics
Composite materials
Semantics
Image quality

Cite this

Sharif, N., Bennamoun, M., White, L. R., & Shah, S. (2018). Learning-based Composite Metrics for Improved Caption Evaluation. In Proceedings of ACL 2018, Student Research Workshop (pp. 14-20). Australia: Association for Computational Linguistics.
Sharif, Naeha ; Bennamoun, Mohammed ; White, Lyndon Rhys ; Shah, Syed. / Learning-based Composite Metrics for Improved Caption Evaluation. Proceedings of ACL 2018, Student Research Workshop. Australia : Association for Computational Linguistics, 2018. pp. 14-20
@inproceedings{e2450e7976b5467397e2d4c4914ccf8d,
title = "Learning-based Composite Metrics for Improved Caption Evaluation",
abstract = "The evaluation of image caption quality is a challenging task, which requires the assessment of two main aspects in a caption: adequacy and fluency. These quality aspects can be judged using a combination of several linguistic features. However, most of the current image captioning metrics focus only on specific linguistic facets, such as the lexical or semantic, and fail to meet a satisfactory level of correlation with human judgements at the sentence-level. We propose a learning-based framework to incorporate the scores of a set of lexical and semantic metrics as features, to capture the adequacy and fluency of captions at different linguistic levels. Our experimental results demonstrate that composite metrics draw upon the strengths of standalone measures to yield improved correlation and accuracy.",
author = "Naeha Sharif and Mohammed Bennamoun and White, {Lyndon Rhys} and Syed Shah",
year = "2018",
language = "English",
pages = "14--20",
booktitle = "Proceedings of ACL 2018, Student Research Workshop",
publisher = "Association for Computational Linguistics",

}

Sharif, N, Bennamoun, M, White, LR & Shah, S 2018, Learning-based Composite Metrics for Improved Caption Evaluation. in Proceedings of ACL 2018, Student Research Workshop. Association for Computational Linguistics, Australia, pp. 14-20, 56th Annual Meeting of Association for Computational Linguistics, Melbourne, Australia, 15/07/18.

Learning-based Composite Metrics for Improved Caption Evaluation. / Sharif, Naeha; Bennamoun, Mohammed; White, Lyndon Rhys; Shah, Syed.

Proceedings of ACL 2018, Student Research Workshop. Australia : Association for Computational Linguistics, 2018. p. 14-20.

Research output: Chapter in Book/Conference paperConference paper

TY - GEN

T1 - Learning-based Composite Metrics for Improved Caption Evaluation

AU - Sharif, Naeha

AU - Bennamoun, Mohammed

AU - White, Lyndon Rhys

AU - Shah, Syed

PY - 2018

Y1 - 2018

N2 - The evaluation of image caption quality is a challenging task, which requires the assessment of two main aspects in a caption: adequacy and fluency. These quality aspects can be judged using a combination of several linguistic features. However, most of the current image captioning metrics focus only on specific linguistic facets, such as the lexical or semantic, and fail to meet a satisfactory level of correlation with human judgements at the sentence-level. We propose a learning-based framework to incorporate the scores of a set of lexical and semantic metrics as features, to capture the adequacy and fluency of captions at different linguistic levels. Our experimental results demonstrate that composite metrics draw upon the strengths of standalone measures to yield improved correlation and accuracy.

AB - The evaluation of image caption quality is a challenging task, which requires the assessment of two main aspects in a caption: adequacy and fluency. These quality aspects can be judged using a combination of several linguistic features. However, most of the current image captioning metrics focus only on specific linguistic facets, such as the lexical or semantic, and fail to meet a satisfactory level of correlation with human judgements at the sentence-level. We propose a learning-based framework to incorporate the scores of a set of lexical and semantic metrics as features, to capture the adequacy and fluency of captions at different linguistic levels. Our experimental results demonstrate that composite metrics draw upon the strengths of standalone measures to yield improved correlation and accuracy.

UR - http://aclweb.org/anthology/P18-3003

UR - https://acl2018.org/call-for-papers/

M3 - Conference paper

SP - 14

EP - 20

BT - Proceedings of ACL 2018, Student Research Workshop

PB - Association for Computational Linguistics

CY - Australia

ER -

Sharif N, Bennamoun M, White LR, Shah S. Learning-based Composite Metrics for Improved Caption Evaluation. In Proceedings of ACL 2018, Student Research Workshop. Australia: Association for Computational Linguistics. 2018. p. 14-20