Entities, Dates, and Languages: Zero-Shot on Historical Texts with T0

Francesco De Toni, Christopher Akiki, Javier de la Rosa, Clémentine Fourrier, Enrique Manjavacas, Stefan Schweter, Daniel van Strien

Research output: Chapter in Book/Conference paperConference paperpeer-review

4 Citations (Scopus)

Abstract

In this work, we explore whether the recently demonstrated zero-shot abilities of the T0 model extend to Named Entity Recognition for out-of-distribution languages and time periods. Using a historical newspaper corpus in 3 languages as test-bed, we use prompts to extract possible named entities. Our results show that a naive approach for prompt-based zero-shot multilingual Named Entity Recognition is error-prone, but highlights the potential of such an approach for historical languages lacking labeled datasets. Moreover, we also find that T0-like models can be probed to predict the publication date and language of a document, which could be very relevant for the study of historical texts.
Original languageEnglish
Title of host publicationProceedings of BigScience Episode #5
Subtitle of host publicationWorkshop on Challenges & Perspectives in Creating Large Language Models
EditorsAngela Fan, Suzana Ilic, Thomas Wolf, Matthias Gallé
Place of PublicationUSA
PublisherAssociation for Computational Linguistics
Pages75-83
Number of pages9
ISBN (Electronic)9781955917261
ISBN (Print)9781955917261
Publication statusPublished - May 2022
EventBigScience Episode #5 – Challenges & Perspectives in Creating Large Language Models: ACL 2022 Workshop -
Duration: 27 May 202227 May 2022

Workshop

WorkshopBigScience Episode #5 – Challenges & Perspectives in Creating Large Language Models
Period27/05/2227/05/22

Fingerprint

Dive into the research topics of 'Entities, Dates, and Languages: Zero-Shot on Historical Texts with T0'. Together they form a unique fingerprint.

Cite this