CLASSIFICATION OF CUSTOMER SENTIMENTS BASED ON ONLINE REVIEWS: COMPARATIVE ANALYSIS OF MACHINE LEARNING AND DEEP LEARNING ALGORITHMS

Vahid Sinap

doi:10.17780/ksujes.1420530

TR EN

ÇEVRİMİÇİ DEĞERLENDİRMELER ÜZERİNDEN MÜŞTERİ DUYGULARININ SINIFLANDIRILMASI: MAKİNE ÖĞRENMESİ VE DERİN ÖĞRENME ALGORİTMALARININ KARŞILAŞTIRMALI

Öz

Geleneksel perakende modellerinden çevrimiçi platformlara geçişte e-ticaretin tüketici davranışlarını dönüştürücü etkisi müşteri duygularını anlamanın önemini artırmıştır. Bu araştırma, çeşitli metin madenciliği ve makine öğrenmesi algoritmalarının duygu analizi ve çevrimiçi değerlendirme verileri bağlamında performanslarını karşılaştırmayı amaçlamaktadır. Bu amaç doğrultusunda Lojistik Regresyon, Naive Bayes, Destek Vektör Makinesi, Rastgele Orman ve AdaBoost olmak üzere toplam beş denetimli sınıflandırma algoritması ve bir derin öğrenme modeli olan CNN Model ile analizler gerçekleştirilmiştir. Çalışmada kullanılan veri seti, bir kadın giyim e-ticaret platformundan elde edilen müşteri değerlendirmelerini içermektedir. Veri setinde ön işlemeler gerçekleştirilerek eksik veriler tamamlanmıştır. Count Vectorizer ve TF-IDF vektörizasyonları yapılarak metinsel verilerin dönüşümü sağlanmıştır. Bunlara ek olarak çeşitli metin ön işleme adımları uygulanmıştır. Araştırmadan elde edilen bulgulara göre müşteri duygularını sınıflandırma bağlamında en etkili algoritmalar AdaBoost ve Naive Bayes algoritmaları olmuştur. Kullanılan vektörizasyon yöntemi açısından önemli bir farklılık tespit edilmemiştir. CNN Model yüksek performans gösterse de modelin eğitimi sırasında aşırı öğrenme tespit edildiği için modelin genellenebilirliği düşük kabul edilmiştir.

Anahtar Kelimeler

CLASSIFICATION OF CUSTOMER SENTIMENTS BASED ON ONLINE REVIEWS: COMPARATIVE ANALYSIS OF MACHINE LEARNING AND DEEP LEARNING ALGORITHMS

Abstract

E-commerce's transformation of consumer behavior has increased the importance of understanding customer emotions, especially in the transition from traditional retail models to online platforms. The proliferation of online shopping has fundamentally changed not only shopping habits but also consumer interactions and purchase decisions. This research aims to compare and analyze the performance of various text mining and machine learning algorithms in the context of sentiment analysis and online review data. For this purpose, analyses were performed with a total of five supervised classification algorithms including Logistic Regression, Naive Bayes, Support Vector Machine, Random Forest, AdaBoost, and a deep learning model, CNN Model. The dataset used in the study includes customer reviews obtained from a women's clothing e-commerce platform. The missing data were completed by pre-processing the dataset. Count Vectorizer and TF-IDF vectorization were performed to transform the textual data. In addition, various text preprocessing steps were applied. According to the findings obtained from the research, AdaBoost and Naive Bayes algorithms were the most effective algorithms in terms of classifying customer sentiments. No significant difference was detected in terms of the vectorization method used. Although the CNN Model showed high performance, the generalizability of the model was considered low because overfitting was detected during the training of the model.

Keywords

References

Agarap, A. F. (2018). Statistical analysis on E-commerce reviews, with sentiment classification using bidirectional recurrent neural network (RNN). arXiv preprint arXiv:1805.03687.
Aizawa, A. (2003). An information-theoretic perspective of TF-IDF measures. Information Processing & Management, 39(1), 45-65.
Alantari, H. J., Currim, I. S., Deng, Y., & Singh, S. (2022). An empirical comparison of machine learning methods for text-based sentiment analysis of online consumer reviews. International Journal of Research in Marketing, 39(1), 1-19.
Alexopoulou, T., Michel, M., Murakami, A., & Meurers, D. (2017). Task effects on linguistic complexity and accuracy: A large‐scale learner corpus analysis employing natural language processing techniques. Language Learning, 67(S1), 180-208.
Angulakshmi, G., & ManickaChezian, R. (2014). An analysis on opinion mining: techniques and tools. International Journal of Advanced Research in Computer and Communication Engineering, 3(7), 2319-5940.
Badaro, G., Baly, R., Hajj, H., Habash, N., & El-Hajj, W. (2014, October). A large scale Arabic sentiment lexicon for Arabic opinion mining. In Proceedings of the EMNLP 2014 workshop on arabic natural language processing (ANLP) (pp. 165-173).
Bafna, P., Pramod, D., & Vaidya, A. (2016, March). Document clustering: TF-IDF approach. In 2016 International Conference on Electrical, Electronics, and Optimization Techniques (ICEEOT) (pp. 61-66). IEEE.
Barik, K., Misra, S., Ray, A. K., & Bokolo, A. (2023). LSTM-DGWO-Based sentiment analysis framework for analyzing online customer reviews. Computational Intelligence and Neuroscience, 2023.

Berrar, D. (2018). Bayes’ theorem and naive Bayes classifier. Encyclopedia of Bioinformatics and Computational Biology: ABC of Bioinformatics, 403, 412.
Biradar, S. H., Gorabal, J. V., & Gupta, G. (2022). Machine learning tool for exploring sentiment analysis on twitter data. Materials Today: Proceedings, 56, 1927-1934.
Brooks, N. (2018). Women’s E-Commerce Clothing Reviews. Kaggle. https://www.kaggle.com/nicapotato/womens-ecommerce-clothing-reviews
Carrigan, M., Moraes, C., & Leek, S. (2011). Fostering responsible communities: A community social marketing approach to sustainable living. Journal of Business Ethics, 100, 515-534.
Carter, J. V., Pan, J., Rai, S. N., & Galandiuk, S. (2016). ROC-ing along: Evaluation and interpretation of receiver operating characteristic curves. Surgery, 159(6), 1638-1645.
Carvalho, D. V., Pereira, E. M., & Cardoso, J. S. (2019). Machine learning interpretability: A survey on methods and metrics. Electronics, 8(8), 832.
Cervantes, J., Garcia-Lamont, F., Rodríguez-Mazahua, L., & Lopez, A. (2020). A comprehensive survey on support vector machine classification: Applications, challenges and trends. Neurocomputing, 408, 189-215.
Chandrasekaran, D., & Tellis, G. J. (2017). A critical review of marketing research on diffusion of new products. Review of Marketing Research, 3, 39-80.
Chawla, N., & Kumar, B. (2022). E-commerce and consumer protection in India: The emerging trend. Journal of Business Ethics, 180(2), 581-604.
Cloutier, N. A., & Japkowicz, N. (2023, December). Fine-tuned generative LLM oversampling can improve performance over traditional techniques on multiclass imbalanced text classification. In 2023 IEEE International Conference on Big Data (BigData) (pp. 5181-5186). IEEE.
Das, A. (2021). Logistic regression. In Encyclopedia of Quality of Life and Well-Being Research (pp. 1-2). Cham: Springer International Publishing.
Demirbilek, M., & Demirbilek, S. Ö. (2023). Sentiment analysis based on google comments with machine learning methods and Amazon Comprehend: The case of a university in Central Anatolia. Journal of University Research, 6(4), 452-461.
Denny, M. J., & Spirling, A. (2018). Text preprocessing for unsupervised learning: Why it matters, when it misleads, and what to do about it. Political Analysis, 26(2), 168-189.
Dey, S., Wasif, S., Tonmoy, D. S., Sultana, S., Sarkar, J., & Dey, M. (2020, February). A comparative study of support vector machine and Naive Bayes classifier for sentiment analysis on Amazon product reviews. In 2020 International Conference on Contemporary Computing and Applications (IC3A) (pp. 217-220). IEEE.
Dogru, N., & Subasi, A. (2018, February). Traffic accident detection using random forest classifier. In 2018 15th learning and technology conference (L&T) (pp. 40-45). IEEE.
Feng, X., Liang, Y., Shi, X., Xu, D., Wang, X., & Guan, R. (2017). Overfitting reduction of text classification based on AdaBELM. Entropy, 19(7), 330.
Georgescu, I., & Kinnunen, J. (2020). Consumer recommendation dynamics in online retail business under logistic regression and naïve Bayes analyses. In Proceedings of the International Conference on Applied Statistics (Vol. 2, No. 1, pp. 120-128).
Guo, J. (2022). Deep learning approach to text analysis for human emotion detection from big data. Journal of Intelligent Systems, 31(1), 113-126.
Han, B., & Baldwin, T. (2011, June). Lexical normalisation of short text messages: Makn sens a# twitter. In Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies (pp. 368-378).
Han, S., & Anderson, C. K. (2020). Customer motivation and response bias in online reviews. Cornell Hospitality Quarterly, 61(2), 142-153.
Hartmann, J., & Netzer, O. (2023). Natural language processing in marketing. In Artificial Intelligence in Marketing (Vol. 20, pp. 191-215). Emerald Publishing Limited.
Hennig-Thurau, T., Wiertz, C., & Feldhaus, F. (2015). Does Twitter matter? The impact of microblogging word of mouth on consumers’ adoption of new movies. Journal of the Academy of Marketing Science, 43, 375-394.
Hossain, M. S., & Rahman, M. F. (2023). Customer sentiment analysis and prediction of insurance products’ reviews using machine learning approaches. FIIB Business Review, 12(4), 386-402.
Hu, W., Gong, Z., & Guo, J. (2010, November). Mining product features from online reviews. In 2010 IEEE 7th International Conference on E-Business Engineering (pp. 24-29). IEEE.
Inácio, M., & Oliveira, H. G. (2024, March). Exploring multimodal models for humor recognition in Portuguese. In Proceedings of the 16th International Conference on Computational Processing of Portuguese (pp. 568-574).
Jararweh, Y., Al-Ayyoub, M., Fakirah, M., Alawneh, L., & Gupta, B. B. (2019). Improving the performance of the needleman-wunsch algorithm using parallelization and vectorization techniques. Multimedia Tools and Applications, 78, 3961-3977.
Kamal, M., & Himel, A. S. (2023). Redefining Modern Marketing: An analysis of AI and NLP's influence on consumer engagement, strategy, and beyond. Eigenpub Review of Science and Technology, 7(1), 203-223.
Kapoor, S., & Banerjee, S. (2021). On the relationship between brand scandal and consumer attitudes: A literature review and research agenda. International Journal of Consumer Studies, 45(5), 1047-1078.
Kowsari, K., Jafari Meimandi, K., Heidarysafa, M., Mendu, S., Barnes, L., & Brown, D. (2019). Text classification algorithms: A survey. Information, 10(4), 150.
Li, C., Zhang, Z., Lee, W. S., & Lee, G. H. (2018). Convolutional sequence to sequence model for human dynamics. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 5226-5234).
Li, H., Bruce, X. B., Li, G., & Gao, H. (2023). Restaurant survival prediction using customer-generated content: An aspect-based sentiment analysis of online reviews. Tourism Management, 96, 104707.
Li, X., Wang, L., & Sung, E. (2005, July). A study of AdaBoost with SVM based weak learners. In Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005. (Vol. 1, pp. 196-201). IEEE.
Li, X., Wang, L., & Sung, E. (2008). AdaBoost with SVM-based component classifiers. Engineering Applications of Artificial Intelligence, 21(5), 785-795.
Lian, H., Lu, C., Li, S., Zhao, Y., Tang, C., & Zong, Y. (2023). A survey of deep learning-based multimodal emotion recognition: speech, text, and face. Entropy, 25(10), 1440.
Maronikolakis, A., & Schütze, H. (2021, April). Multidomain pretrained language models for green NLP. In Proceedings of the Second Workshop on Domain Adaptation for NLP (pp. 1-8).
Mariani, M., & Borghi, M. (2021). Are environmental-related online reviews more helpful? A big data analytics approach. International Journal of Contemporary Hospitality Management, 33(6), 2065-2090.
Minaee, S., Kalchbrenner, N., Cambria, E., Nikzad, N., Chenaghlu, M., & Gao, J. (2021). Deep learning--based text classification: a comprehensive review. ACM Computing Surveys (CSUR), 54(3), 1-40.
Mohammad, S. M. (2016). Sentiment analysis: Detecting valence, emotions, and other affectual states from text. In Emotion measurement (pp. 201-237). Woodhead Publishing.
Najafabadi, M. M., Villanustre, F., Khoshgoftaar, T. M., Seliya, N., Wald, R., & Muharemagic, E. (2015). Deep learning applications and challenges in big data analytics. Journal of Big Data, 2(1), 1-21.
Obiedat, R., Qaddoura, R., Ala’M, A. Z., Al-Qaisi, L., Harfoushi, O., Alrefai, M. A., & Faris, H. (2022). Sentiment analysis of customers’ reviews using a hybrid evolutionary svm-based approach in an imbalanced data distribution. IEEE Access, 10, 22260-22273.
Patel, A., Oza, P., & Agrawal, S. (2023). Sentiment analysis of customer feedback and reviews for airline services using language representation model. Procedia Computer Science, 218, 2459-2467.
Pisner, D. A., & Schnyer, D. M. (2020). Support vector machine. In Machine learning (pp. 101-121). Academic Press.
Pradhan, V. M., Vala, J., & Balani, P. (2016). A survey on sentiment analysis algorithms for opinion mining. International Journal of Computer Applications, 133(9), 7-11.
Qaiser, S., & Ali, R. (2018). Text mining: use of TF-IDF to examine the relevance of words to documents. International Journal of Computer Applications, 181(1), 25-29.
Racherla, P., & Friske, W. (2012). Perceived ‘usefulness’ of online consumer reviews: An exploratory investigation across three services categories. Electronic Commerce Research and Applications, 11(6), 548-559.
Rain, C. (2013). Sentiment analysis in Amazon reviews using probabilistic machine learning. Swarthmore College, 42.
Ramadhan, F. A., Ruslan, R. R. P., & Zahra, A. (2023). Sentiment analysis of e-commerce product reviews for content interaction using machine learning. Cakrawala Repositori IMWI, 6(1), 207-220.
Robertson, S. (2004). Understanding inverse document frequency: on theoretical arguments for IDF. Journal of Documentation, 60(5), 503-520.
Rosário, A., & Raimundo, R. (2021). Consumer marketing strategy and e-commerce in the last decade: a literature review. Journal of Theoretical and Applied Electronic Commerce Research, 16(7), 3003-3024.
Rygielski, C., Wang, J. C., & Yen, D. C. (2002). Data mining techniques for customer relationship management. Technology in Society, 24(4), 483-502.
Sarker, I. H. (2021). Machine learning: Algorithms, real-world applications and research directions. SN Computer Science, 2(3), 160.
Shanthi, R., & Desti, K. (2015). Consumers' perception on online shopping. Journal of Marketing and Consumer Research, 13, 14-21.
Singh, J. P., Irani, S., Rana, N. P., Dwivedi, Y. K., Saumya, S., & Roy, P. K. (2017). Predicting the “helpfulness” of online consumer reviews. Journal of Business Research, 70, 346-355.
Sylvester, E. V., Bentzen, P., Bradbury, I. R., Clément, M., Pearce, J., Horne, J., & Beiko, R. G. (2018). Applications of random forest feature selection for fine‐scale genetic population assignment. Evolutionary Applications, 11(2), 153-165.
Tran, D. D., Nguyen, T. T. S., & Dao, T. H. C. (2022). Sentiment analysis of movie reviews using machine learning techniques. In Proceedings of Sixth International Congress on Information and Communication Technology: ICICT 2021, London, Volume 1 (pp. 361-369). Springer Singapore.
Turki, T., & Roy, S. S. (2022). Novel hate speech detection using word cloud visualization and ensemble learning coupled with count vectorizer. Applied Sciences, 12(13), 6611.
Vijayarani, S., & Janani, R. (2016). Text mining: open source tokenization tools-an analysis. Advanced Computational Intelligence: An International Journal (ACII), 3(1), 37-47.
Wani, T. A., & Ali, S. W. (2015). Innovation diffusion theory. Journal of General Management Research, 3(2), 101-118.
Weiss, S. M., Indurkhya, N., Zhang, T., & Damerau, F. (2010). Text mining: predictive methods for analyzing unstructured information. Springer Science & Business Media.
Wyner, A. J., Olson, M., Bleich, J., & Mease, D. (2017). Explaining the success of adaboost and random forests as interpolating classifiers. The Journal of Machine Learning Research, 18(1), 1558-1590.
Xia, R., Zong, C., & Li, S. (2011). Ensemble of feature sets and classification algorithms for sentiment classification. Information Sciences, 181(6), 1138-1152.
Xiong, H., Pandey, G., Steinbach, M., & Kumar, V. (2006). Enhancing data analysis with noise removal. IEEE Transactions on Knowledge and Data Engineering, 18(3), 304-319.
Zhang, F., Fleyeh, H., Wang, X., & Lu, M. (2019). Construction site accident analysis using text mining and natural language processing techniques. Automation in Construction, 99, 238-248.
Zhang, J., Lu, X., & Liu, D. (2021). Deriving customer preferences for hotels based on aspect-level sentiment analysis of online reviews. Electronic Commerce Research and Applications, 49, 101094.
Zhao, H., Liu, Z., Yao, X., & Yang, Q. (2021). A machine learning-based sentiment analysis of online product reviews with a novel term weighting and feature selection approach. Information Processing & Management, 58(5), 102656.
Zhou, J., & Troyanskaya, O. G. (2015). Predicting effects of noncoding variants with deep learning–based sequence model. Nature Methods, 12(10), 931-934.
Zhou, L., Pan, S., Wang, J., & Vasilakos, A. V. (2017). Machine learning on big data: Opportunities and challenges. Neurocomputing, 237, 350-361.
Zhuang, M., Cui, G., & Peng, L. (2018). Manufactured opinions: The effect of manipulating online product reviews. Journal of Business Research, 87, 24-35.

Details

Primary Language

English

Subjects

Data Mining and Knowledge Discovery , Natural Language Processing

Journal Section

Research Article

Authors

Vahid Sinap ^*
0000-0002-8734-9509
Türkiye

Publication Date

September 3, 2024

Submission Date

January 16, 2024

Acceptance Date

July 31, 2024

Published in Issue

Year 2024 Volume: 27 Number: 3

DOI

https://doi.org/10.17780/ksujes.1420530

IZ

https://izlik.org/JA63KH88LM

Cite

RIS / Bibtex

APA

Sinap, V. (2024). CLASSIFICATION OF CUSTOMER SENTIMENTS BASED ON ONLINE REVIEWS: COMPARATIVE ANALYSIS OF MACHINE LEARNING AND DEEP LEARNING ALGORITHMS. Kahramanmaraş Sütçü İmam Üniversitesi Mühendislik Bilimleri Dergisi, 27(3), 779-799. https://doi.org/10.17780/ksujes.1420530

Cited By

Multilingual sentiment analysis in e-commerce customer reviews using GPT and deep learning-based weighted-ensemble model

International Journal of Cognitive Computing in Engineering

https://doi.org/10.1016/j.ijcce.2025.10.003