A Comparative Study of Deep Learning and Transformer Models for Twitter Sentiment Analysis
DOI:
https://doi.org/10.61467/2007.1558.2026.v17i1.1241Keywords:
Sentiment Analysis, CNN, BiLSTM, GRU, DistilBERT, Deep LearningAbstract
In this work, we investigate sentiment classification on Twitter using the Sentiment140 dataset and compare traditional deep learning approaches, including CNN, BiLSTM, and GRU, with a lightweight transfer learning model, DistilBERT. The results indicate that the CNN, BiLSTM, and GRU models achieve accuracy levels ranging from 71% to 79%, whereas the fine-tuned DistilBERT model attains an accuracy of 86% with an F1-score of 0.86. These findings suggest that, while conventional neural models tend to focus on more local patterns, lightweight transformer-based architectures are better able to capture[HEDGING-END] broader linguistic representations, while still maintaining relatively efficient computational performance.
Overall, this study offers a systematic and practical comparison of sentiment classification models, supported by a clearly defined methodology, empirical results, and corresponding tables and figures. The analysis may provide useful guidance for researchers and practitioners interested in sentiment analysis applications for social media data.
Smart citations: https://scite.ai/reports/10.61467/2007.1558.2026.v17i1.1241
References
Ahmed, S., & Khan, T. (2023). Transformer-based sentiment classification in social media. IEEE Access.
Alharbi, A., Khan, M., & Hussain, S. (2022). Multilingual sentiment analysis using XLM-R transformers. Applied Intelligence.
Brown, P., & Liu, S. (2025). Ethical challenges in sentiment analysis with deep learning. AI Ethics Review.
Chen, L., Zhang, R., & Li, P. (2022). Domain-adaptive transformers for financial sentiment analysis. Expert Systems with Applications.
Ghosh, S., & Banerjee, A. (2024). Explainable sentiment analysis with transformer models. Artificial Intelligence Review.
Hafeez, M., Hussain, N., Qasim, A., Zain, M., Mehak, G., Kolesnikova, O., … & Gelbukh, A. (2025, October). Sarcasm detection in Roman Urdu text: A comprehensive study using machine learning and large language models. In Mexican International Conference on Artificial Intelligence (pp. 245–254). Springer Nature Switzerland.
Hussain, N., Qasim, A., Liaquat, F., Mehak, G., Meque, A. G. M., Usman, M., … & Gelbukh, A. (2025, October). Toward bias-aware and efficient offensive language detection using QLoRA-optimized LLaMA and GPT models. In Mexican International Conference on Artificial Intelligence (pp. 206–217). Springer Nature Switzerland.
Hussain, N., Qasim, A., Mehak, G., Kolesnikova, O., Gelbukh, A., & Sidorov, G. (2025). ORUD-Detect: A comprehensive approach to offensive language detection in Roman Urdu using hybrid machine learning–deep learning models with embedding techniques. Information, 16(2), 139.
Hussain, N., Qasim, A., Mehak, G., Kolesnikova, O., Gelbukh, A., & Sidorov, G. (2025). Hybrid machine learning and deep learning approaches for insult detection in Roman Urdu text. AI, 6(2), 33.
Hussain, N., Qasim, A., Mehak, G., Zain, M., Hafeez, M., & Sidorov, G. (2025). Fine-tuning large language models with QLoRA for offensive language detection in Roman Urdu–English code-mixed text. arXiv. https://arxiv.org/abs/2510.03683
Hussain, N., Qasim, A., Mehak, G., Zain, M., Sidorov, G., Gelbukh, A., & Kolesnikova, O. (2025). Multi-level depression severity detection with deep transformers and enhanced machine learning techniques. AI, 6(7), 157.
Kumar, A., & Joshi, R. (2021). Hybrid deep learning models for sentiment analysis in noisy social media data. Information Processing & Management.
National Center for Biotechnology Information. (n.d.). NCBI. Retrieved March 15, 2024, from http://www.ncbi.nlm.nih.gov
Rahman, M., Alam, S., & Islam, R. (2023). Comparative evaluation of deep learning and transformers for sentiment analysis. Future Internet.
Sun, C., Qiu, X., Xu, Y., & Huang, X. (2020). How to fine-tune BERT for text classification? In Lecture Notes in Computer Science.
Wang, Q., Liu, Z., & Gao, Y. (2023). Sentiment analysis of COVID-19 vaccine discourse using transformers. Social Network Analysis and Mining.
Zhang, Y., Li, X., & Zhao, H. (2021). Sentiment analysis on Twitter using RoBERTa and traditional classifiers. Journal of NLP Research.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 International Journal of Combinatorial Optimization Problems and Informatics
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
