Pheonix at SemEval-2020 Task 5: Masking the Labels Lubricates Models for Sequence Labeling

Pouria Babvey, Dario Borrelli, Yutong Zhao, Carlo Lipizzi

    Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

    3 Scopus citations

    Abstract

    This paper presents the deep-learning model that is submitted to the SemEval-2020 Task 5 competition: “Detecting Counterfactuals”. We participated in both Subtask1 and Subtask2. The model proposed in this paper ranked 2nd in Subtask2: “Detecting antecedent and consequence”. Our model approaches the task as a sequence labeling. The architecture is built on top of BERT; and a multi-head attention layer with label masking is used to benefit from the mutual information between nearby labels. Also, for prediction, a multi-stage algorithm is used in which the model finalize some predictions with higher certainty in each step and use them in the following. Our results show that masking the labels not only is an efficient regularization method but also improves the accuracy of the model compared with other alternatives like CRF. Label masking can be used as a regularization method in sequence labeling. Also, it improves the performance of the model by learning the specific patterns in the target variable.

    Original languageEnglish
    Title of host publication14th International Workshops on Semantic Evaluation, SemEval 2020 - co-located 28th International Conference on Computational Linguistics, COLING 2020, Proceedings
    EditorsAurelie Herbelot, Xiaodan Zhu, Alexis Palmer, Nathan Schneider, Jonathan May, Ekaterina Shutova
    Pages677-682
    Number of pages6
    ISBN (Electronic)9781952148316
    StatePublished - 2020
    Event14th International Workshops on Semantic Evaluation, SemEval 2020 - Barcelona, Spain
    Duration: 12 Dec 202013 Dec 2020

    Publication series

    Name14th International Workshops on Semantic Evaluation, SemEval 2020 - co-located 28th International Conference on Computational Linguistics, COLING 2020, Proceedings

    Conference

    Conference14th International Workshops on Semantic Evaluation, SemEval 2020
    Country/TerritorySpain
    CityBarcelona
    Period12/12/2013/12/20

    Fingerprint

    Dive into the research topics of 'Pheonix at SemEval-2020 Task 5: Masking the Labels Lubricates Models for Sequence Labeling'. Together they form a unique fingerprint.

    Cite this