تحلیل احساسات رمزارزها با یادگیری انتقالی شات صفر

جهان بین, کیا; زارع چاهوکی, محمدعلی

doi:10.22052/scj.2025.255169.1258

تحلیل احساسات رمزارزها با یادگیری انتقالی شات صفر

نوع مقاله : مقاله پژوهشی

نویسندگان

گروه کامپیوتر و نرم‌افزار، دانشکده مهندسی کامپیوتر، دانشگاه یزد، یزد، ایران.

10.22052/scj.2025.255169.1258

چکیده

تحلیل احساسات جنبه‌محور ‌توئیت‌ها با شبکه‌های عصبی عمیق بسیار مورد توجه است. با این حال، این شبکه‌‌ها برای کارایی مطلوب نیازمند دادگان غنی می‌باشند. بنابراین، در این مطالعه از یادگیری انتقالی عمیق شات صفر و دادگان میان‌دامنه‌ای برای افزایش غنای منابع آموزشی در دامنه هدف استفاده شده است. در این پژوهش، ابتدا توئیت‌های تاثیرگذاران رمزارزها جمع‌آوری شده و سپس، از ترکیب شبکه‌ عصبی عمیق، لایه توجه و شبکه‌ پیش‌آموزش‌دیده دستیلبرت برای تحلیل احساسات جنبه‌محور استفاده شده است. نقشه حرارتی لایه توجه نشان می‌دهد استفاده از این لایه بعد از مدل‌های عمیق توانسته به برجسته‌سازی کلمات جنبه کمک کند. در یادگیری انتقالی شات صفر، مدل هیچ نمونه‌ای از داده‌های هدف دارای برچسب را در اختیار ندارد و آموزش با دادگان میان دامنه‌ای انجام می‌شود. علاوه بر این، استفاده از دادگان میان‌دامنه‌ای و انتخاب بهینه آنها با از ضریب شباهت پیرسون در شرایط کمبود داده غنی سبب کاهش انتقال منفی شده و همچنین حساسیت مدل را به محتوی زمینه متن کمک می‌کند. همچنین، برای حل مشکل دادگان نامتوازن از رویکرد نمونه‌برداری بیش از حد اقلیت مصنوعی استفاده شده است. یافته‌های تجربی نشان می‌دهد، مدل در مقایسه با پژوهش‌های پیشین، به صورت میانگین دقت و F1 را بر روی مجموعه ‌داده آزمون SemEval در حدود 2 درصد افزایش داده و میانگین نرخ ROC-AUC به 86.35 درصد رسیده است. همچنین، در این پژوهش از نرخ یادگیری وفق‌پذیر استفاده شده که سبب همگرایی سریع‌تر مدل می‌شود. انتخاب ابرپارامترهای شبکه نیز‌ با روش گریدی صورت گرفته که موجب وفق‌پذیری بیشتر مدل با دامنه هدف شده است.

کلیدواژه‌ها

موضوعات

پردازش زبان طبیعی

عنوان مقاله [English]

Sentiment analysis of cryptocurrencies with Zero-shot transfer learning

نویسندگان [English]

Kia Jahanbin
Mohammad Ali Zare Chahooki

Department of Computer Engineering, Yazd University, Yazd, Iran.

چکیده [English]

Aspect-based sentiment analysis (ABSA) of tweets using deep neural networks has garnered significant attention. However, these networks generally perform best with access to extensive, high-quality data. This research leverages zero-shot deep transfer learning and cross-domain data to enrich the educational resources available within the target domain. The study began by gathering tweets from cryptocurrency influencers, followed by employing a combination of a deep neural network, an attention mechanism, and the pre-trained DistillBERT model for aspect-based sentiment analysis. The attention layer's heat map indicates that integrating this layer with deep models enhances the identification of aspect-specific words. In zero-shot transfer learning, the model operates without labelled target data samples, relying exclusively on cross-domain data for training. Additionally, using cross-domain data with optimal selection based on Pearson's similarity coefficient helps minimize negative transfer under low-data conditions, improving the model's sensitivity to textual context. To address data imbalance, the Synthetic Minority Over-Sampling Technique (SMOTE) was applied. Experimental results indicate a 2% improvement in average accuracy and F1 score on the SemEval benchmark dataset, with the ROC-AUC reaching 86.35% relative to previous studies. Furthermore, we employed an adaptive learning rate to expedite model convergence and used Grid Search to optimize the network's hyperparameters, enhancing its adaptability to the target domain.

کلیدواژه‌ها [English]

Cryptocurrency aspect-briented sentiment analysis
Zero-shot transfer learning
Pre-trained deep neural networks
attention layer
Grid search
Adaptive learning rate

مراجع

[1] J. Chai and A. Li, “Deep learning in natural language processing: A state-of-the-art survey,” in Proc. 2019 Int. Conf. Mach. Learn. Cybern. (ICMLC), 2019, doi: 10.1109/ICMLC48188.2019.8949185.
[2] F. Zare Mehrjardi, M. Yazdian-Dehkordi, and A. Latif, “Evaluating classical machine learning and deep-learning methods in sentiment analysis of Persian telegram message,” Soft Comput. J., vol. 11, no. 1, pp. 88-105, 2022, doi: 10.22052/scj.2023.246553.1077 [In Persian].
[3] O. Khalaf Beigi, S.A. Bashiri Mosavi, and S. Gharloghi, “Applying Character-Level Neural Network-Based Sentiment Analysis Model on Persian Comments of the Social Media-Online Store Platforms,” Soft Comput. J., vol. 11, no. 2, pp. 118-133, 2023, doi: 10.22052/scj.2023.248311.1094 [In Persian].
[4] F. Pourgholamali, M. Kahani, and E. Asgarian, “Exploiting Big Data Technology for Opinion Mining,” Soft Comput. J., vol. 9, no. 1, pp. 26-39, 2020, doi: 10.22052/scj.2021.111450 [In Persian].
[5] K.W. Church, Z. Chen, and Y. Ma, “Emerging trends: A gentle introduction to fine-tuning,” Nat. Lang. Eng., vol. 27, no. 6, pp. 763-778, 2021, doi: 10.1017/S1351324921000322.
[6] K. Jahanbin and M.A.Z. Chahooki, “Database of Twitter influencers in cryptocurrency (2021-2023) with sentiments,” Research Square, 2023, doi: 10.21203/rs.3.rs-3192598/v1.
[7] D. Elreedy, A.F. Atiya, and F. Kamalov, “A theoretical distribution analysis of synthetic minority oversampling technique (SMOTE) for imbalanced learning,” Mach. Learn., vol. 113, no. 7, pp. 4903-4923, 2024, doi: 10.1007/s10994-022-06296-4.
[8] G. Lu, Y. Liu, J. Wang, and H. Wu, “CNN-BiLSTM-Attention: A multi-label neural classifier for short texts with a small set of labels,” Inf. Process. Manag., vol. 60, no. 3, p. 103320, 2023, doi: 10.1016/j.ipm.2023.103320.
[9] H.N. Serere, B. Resch, and C.R. Havas, “Enhanced geocoding precision for location inference of tweet text using spaCy, Nominatim and Google Maps. A comparative analysis of the influence of data selection,” PLoS One, vol. 18, no. 3, p. e0282942, 2023, doi: 10.1371/journal.pone.0282942.
[10] M. Isnan, G.N. Elwirehardja, and B. Pardamean, “Sentiment analysis for TikTok review using VADER sentiment and SVM model,” Procedia Comput. Sci., vol. 227, pp. 168-175, 2023, doi: 10.1016/j.procs.2023.10.514.
[11] M. Nair, L.A. Abd-Elmegid, and M.I. Marie, “Sentiment analysis model for cryptocurrency tweets using different deep learning techniques,” J. Intell. Syst., vol. 33, no. 1, 2024, doi: 10.1515/jisys-2023-0085.
[12] T. Pano and R. Kashef, “A complete VADER-based sentiment analysis of bitcoin (BTC) tweets during the era of COVID-19,” Big Data Cogn. Comput., vol. 4, no. 4, p. 33, 2020, doi: 10.3390/bdcc4040033.
[13] A.K. Jha and N.K. Verma, “Social media sustainability communication: An analysis of firm behaviour and stakeholder responses,” Inf. Syst. Front., vol. 25, no. 2, pp. 723-742, 2023, doi: 10.1007/s10796-022-10257-6.
[14] N.K. Singh, D.S. Tomar, and A.K. Sangaiah, “Sentiment analysis: A review and comparative analysis over social media,” J. Ambient Intell. Humaniz. Comput., vol. 11, no. 1, pp. 97-117, 2020, doi: 10.1007/s12652-018-0862-8.
[15] L. Yue, W. Chen, X. Li, W. Zuo, and M. Yin, “A survey of sentiment analysis in social media,” Knowl. Inf. Syst., vol. 60, no. 2, pp. 617-663, 2019, doi: 10.1007/s10115-018-1236-4.
[16] Z. Nanli, Z. Ping, L. Weiguo, and C. Meng, “Sentiment analysis: A literature review,” in Proc. 2012 Int. Symp. Manag. Technol. (ISMOT), 2012, pp. 572-576, doi: 10.1109/ISMOT.2012.6679538.
[17] H. Liu, I. Chatterjee, M. Zhou, X.S. Lu, and A. Abusorrah, “Aspect-based sentiment analysis: A survey of deep learning methods,” IEEE Trans. Comput. Soc. Syst., vol. 7, no. 6, pp. 1358-1375, 2020, doi: 10.1109/TCSS.2020.3033302.
[18] Z. Zhang, P. Cui, and W. Zhu, “Deep learning on graphs: A survey,” IEEE Trans. Knowl. Data Eng., vol. 34, no. 1, pp. 249-270, 2022, doi: 10.1109/TKDE.2020.2981333.
[19] A. Shrestha and A. Mahmood, “Review of deep learning algorithms and architectures,” IEEE Access, vol. 7, pp. 53040-53065, 2019, doi: 10.1109/ACCESS.2019.2912200.
[20] Y. Yu, X. Si, C. Hu, and J. Zhang, “A review of recurrent neural networks: LSTM cells and network architectures,” Neural Comput., vol. 31, no. 7, pp. 1235-1270, 2019, doi: 10.1162/neco_a_01199.
[21] N. Agarwal, A. Sondhi, K. Chopra, and G. Singh, “Transfer learning: Survey and classification,” in Smart Innovations in Communication and Computational Sciences, Singapore: Springer, 2021, pp. 145-155, doi: 10.1007/978-981-15-5345-5_13.
[22] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional Transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018, doi: 10.48550/arXiv.1810.04805.
[23] I. Staliunaite and I. Iacobacci, “Compositional and lexical semantics in RoBERTa, BERT and DistilBERT: A case study on CoQA,” in Proc. 2020 Conf. Empirical Methods Nat. Lang. Process. (EMNLP), 2020, doi: 10.48550/arXiv.2009.08257.
[24] A.F. Adoma, N.-M. Henry, and W. Chen, “Comparative analyses of Bert, Roberta, distilbert, and xlnet for text-based emotion recognition,” in Proc. 2020 17th Int. Comput. Conf. Wavelet Active Media Technol. Inf. Process. (ICCWAMTIP), 2020, doi: 10.1109/ICCWAMTIP51612.2020.9317379.
[25] M. Suhasini and B. Srinivasu, “Emotion detection framework for twitter data using supervised classifiers,” in Advances in Intelligent Systems and Computing, Singapore: Springer, 2020, pp. 565-576, doi: 10.1007/978-981-15-1097-7_47.
[26] J. Abraham, D.W. Higdon, J. Nelson, and J. Ibarra, “Cryptocurrency price prediction using tweet volumes and sentiment analysis,” SMU Data Sci. Rev., vol. 1, no. 3, p. 1, 2018.
[27] S. Biswas, M. Pawar, S. Badole, N. Galande, and S. Rathod, “Cryptocurrency price prediction using neural networks and deep learning,” in Proc. 2021 7th Int. Conf. Adv. Comput. Commun. Syst. (ICACCS), 2021, vol. 1, pp. 408-413, doi: 10.1109/ICACCS51430.2021.9441872.
[28] X. Huang et al., “LSTM based sentiment analysis for cryptocurrency prediction,” in Proc. Database Syst. Adv. Appl. (DASFAA), 2021, pp. 617-621, doi: 10.1007/978-3-030-73200-4_47.
[29] C. Hutto and E. Gilbert, “Vader: A parsimonious rule-based model for sentiment analysis of social media text,” in Proc. Int. AAAI Conf. Web Social Media, 2014, vol. 8, no. 1, pp. 216-225, doi: 10.1609/icwsm.v8i1.14550.
[30] S. Oikonomopoulos, K. Tzafilkou, D. Karapiperis, and V. Verykios, “Cryptocurrency price prediction using social media sentiment analysis,” in Proc. 2022 13th Int. Conf. Inf. Intell. Syst. Appl. (IISA), 2022, pp. 1-8, doi: 10.1109/IISA56318.2022.9904351.
[31] G.C. Habek, M.A. Tocoglu, and A. Onan, “Bi-directional CNN-RNN architecture with group-wise enhancement and attention mechanisms for cryptocurrency sentiment analysis,” Appl. Artif. Intell., vol. 36, no. 1, 2022, doi: 10.1080/08839514.2022.2145641.
[32] I. Moudhich and A. Fennan, “Graph embedding approach to analyze sentiments on cryptocurrency,” Int. J. Electr. Comput. Eng. (IJECE), vol. 14, no. 1, p. 690, 2024, doi: 10.11591/ijece.v14i1.pp690-697.
[33] N. Aslam, F. Rustam, E. Lee, P.B. Washington, and I. Ashraf, “Sentiment analysis and emotion detection on cryptocurrency related tweets using ensemble LSTM-GRU model,” IEEE Access, vol. 10, pp. 39313-39324, 2022, doi: 10.1109/ACCESS.2022.3165621.
[34] J. Davchev, K. Mishev, I. Vodenska, L. Chitkushev, and D. Trajanov, “Bitcoin price prediction using transfer learning on financial micro-blogs,” in Proc. 16th Annu. Int. Conf. Comput. Sci. Educ. Comput. Sci., 2020.
[35] P. Zhao, L. Hou, and O. Wu, “Modeling sentiment dependencies with graph convolutional networks for aspect-level sentiment classification,” Knowl. Based Syst., vol. 193, p. 105443, 2020, doi: 10.1016/j.knosys.2019.105443.
[36] A. Zunic, P. Corcoran, and I. Spasic, “Aspect-based sentiment analysis with graph convolution over syntactic dependencies,” Artif. Intell. Med., vol. 119, p. 102138, 2021, doi: 10.1016/j.artmed.2021.102138.
[37] Y. Lv et al., “Aspect-level sentiment analysis using context and aspect memory network,” Neurocomputing, vol. 428, pp. 195-205, 2021, doi: 10.1016/j.neucom.2020.11.049.
[38] N. Habbat, H. Anoun, and L. Hassouni, “Combination of GRU and CNN deep learning models for sentiment analysis on French customer reviews using XLNet model,” IEEE Eng. Manag. Rev., vol. 51, no. 1, pp. 41-51, 2022, doi: 10.1109/EMR.2022.3208818.
[39] M.M. Kabir, Z.A. Othman, M.R. Yaakub, and S. Tiun, “Hybrid syntax dependency with lexicon and logistic regression for aspect-based sentiment analysis,” Int. J. Adv. Comput. Sci. Appl., vol. 14, no. 10, 2023, doi: 10.14569/IJACSA.2023.0141085.
[40] B. Huang et al., “Aspect-level sentiment analysis with aspect-specific context position information,” Knowl. Based Syst., vol. 243, p. 108473, 2022, doi: 10.1016/j.knosys.2022.108473.
[41] X. Zhu, L. Zhu, J. Guo, S. Liang, and S. Dietze, “GL-GCN: Global and Local Dependency Guided Graph Convolutional Networks for aspect-based sentiment classification,” Expert Syst. Appl., vol. 186, p. 115712, 2021, doi: 10.1016/j.eswa.2021.115712.
[42] S. Chumakov, A. Kovantsev, and A. Surikov, “Generative approach to aspect based sentiment analysis with GPT language models,” Procedia Comput. Sci., vol. 229, pp. 284-293, 2023, doi: 10.1016/j.procs.2023.12.030.
[43] P. Liashchynskyi, “Grid search, random search, genetic algorithm: A big comparison for NAS,” arXiv preprint arXiv:1912.06059, 2019, doi: 10.48550/arXiv.1912.06059.
[44] K. Nakamura, B. Derbel, K.-J. Won, and B.-W. Hong, “Learning-rate annealing methods for deep neural networks,” Electronics, vol. 10, no. 16, p. 2029, 2021, doi: 10.3390/electronics10162029.
[45] G. D’Aniello, M. Gaeta, and I. La Rocca, “KnowMIS-ABSA: An overview and a reference model for applications of sentiment analysis and aspect-based sentiment analysis,” Artif. Intell. Rev., vol. 55, no. 7, pp. 5543-5574, 2022, doi: 10.1007/s10462-021-10134-9.
[46] A.J. Amadeo, J.G. Siento, T.A. Eikwine, Diana, and I.H. Parmonangan, “Temporal Fusion Transformer for multi horizon bitcoin price forecasting,” in Proc. 2023 IEEE 9th Inf. Technol. Int. Seminar (ITIS), 2023, doi: 10.1109/ITIS59651.2023.10420330.
[47] A.Y. Nouira, M. Bouchakwa, and M. Amara, “Role of social networks and machine learning techniques in cryptocurrency price prediction: A survey,” Soc. Netw. Anal. Min., vol. 14, no. 1, 2024, doi: 10.1007/s13278-024-01316-8.
[48] G. Serafini et al., “Sentiment-driven price prediction of the bitcoin based on statistical and deep learning approaches,” in Proc. 2020 Int. Joint Conf. Neural Networks (IJCNN), 2020, doi: 10.1109/IJCNN48605.2020.9206704.
[49] N. Elhadad et al., “SemEval-2015 Task 14: Analysis of clinical text,” in Proc. 9th Int. Worksh. Semantic Eval. (SemEval), 2015, doi: 10.18653/v1/S15-2051.
[50] M. Pontiki et al., “SemEval-2016 Task 5: Aspect based sentiment analysis,” in Proc. 10th Int. Worksh. Semantic Eval. (SemEval), 2016, doi: 10.18653/v1/S16-1002.
[51] M. Rhanoui, M. Mikram, S. Yousfi, and S. Barzali, “A CNN-BiLSTM model for document-level sentiment analysis,” Mach. Learn. Knowl. Extr., vol. 1, no. 3, pp. 832-847, 2019, doi: 10.3390/make1030048.
[52] H. Wan et al., “Target-aspect-sentiment joint detection for aspect-based sentiment analysis,” in Proc. AAAI Conf. Artif. Intell., 2020, vol. 34, no. 5, pp. 9122-9129, doi: 10.1609/aaai.v34i05.6447.
[53] Z. Hu, Z. Wang, Y. Wang, and A.-H. Tan, “MSRL-Net: A multi-level semantic relation-enhanced learning network for aspect-based sentiment analysis,” Expert Syst. Appl., vol. 217, p. 119492, 2023, doi: 10.1016/j.eswa.2022.119492.