Low resource named entity recognition using contextual word representation and neural cross-lingual knowledge transfer

Low resource Named Entity Recognition can be solved by transferring knowledge from a high to a low-resource language with shared multilingual embedding spaces. In this paper, we focus on the extreme low-resource NER scenario of unsupervised cross-lingual knowledge transfer, where no labelled training data or parallel corpus is available. We apply word-alignment with the contextualised word embedding and propose an efficient cross-lingual centroid-based space translation mechanism for contextual embedding. We found that the proposed alignment mechanism works well between different languages, compared to current state-of-the-art models. Moreover, word order differences is another problem to be resolved in cross-lingual NER. We alleviate this issue by incorporating a transformer, which relies entirely on an attention mechanism to draw global dependency between input and output. Our method was evaluated against state-of-the-art results, and it indicate that our approach was better in terms of the performance and the amount of resources.