Exploring BERT-Based Pretrained Models for Polarity Analysis of Tweets in Spanish

Abstract

This paper reviews the implementation of three pre-trained models based on BERT (“bert-base-multilingual-cased”, “IIC/beto-base-spanish-sqac” and “MarcBrun/ixambert-finetuned-squad-eu-en”) to solve tasks 1.1 and 1.2 of “Workshop on Semantic Analysis at SEPLN 2020” (TASS 2020), these tasks consist of the polarity analysis of tweets in Spanish from different Spanish-speaking countries. The proposed models are evaluated individually by pre-processing and replacing synonyms. This research is carried out to find the points to improve in the polarity analysis of tweets (tweets), mainly in how the pre-trained models interpret words that are not in their vocabulary due to variations in the language, regional expressions, misspellings, and use of emojis.