ÖNERİ SİSTEMLERİNDE KULLANILAN PERFORMANS METRİKLERİNİN FİLTRELEME TEKNOLOJİLERİNE GÖRE DEĞERLENDİRİLMESİ: İŞ ÖNERİ SİSTEMLERİ ALANI ÜZERİNE BİR ARAŞTIRMA ÇALIŞMASI

Selin Bitirim; Duygu Çelik Ertuğrul

doi:10.17780/ksujes.1410926

TR EN

ÖNERİ SİSTEMLERİNDE KULLANILAN PERFORMANS METRİKLERİNİN FİLTRELEME TEKNOLOJİLERİNE GÖRE DEĞERLENDİRİLMESİ: İŞ ÖNERİ SİSTEMLERİ ALANI ÜZERİNE BİR ARAŞTIRMA ÇALIŞMASI

Öz

Tavsiye Sistemleri (Recommendation Systems—RSs) sayesinde hemen hemen her sektörde (ör. e-ticaret, eğitim, eğlence, sağlık, insan kaynakları, reklamcılık, vb.) mevcut süreçlerin/operasyonların etkin bir biçimde yürütülebilmesi ve kullanıcının ilgisini çekebilecek öğelere öncelik verilmesi mümkün hale gelmiştir. RS'lerin katkısı ile, sektörel süreçlerin/hizmetlerin etkin şekilde yönetilmesi ve kullanıcılara kişiselleştirilmiş sonuçlar üretilmesi mümkündür. Bu çalışmada, RS ile ilgili araştırmaların gözden geçirilmesi, filtreleme teknikleri taksonomisinin ortaya çıkarılması ve geniş çapta rastlanan performans metriklerinin tespiti amaçlanmaktadır. Ayrıca, İnsan Kaynakları (İK) yönetiminin olmazsa olmazı olan İş Tavsiye Sistemleri bu çalışmada, araştırma sahası olarak seçilmiş olup performans metriklerinin ve öğe filtreleme yaklaşımlarının belirlenmesi planlanmıştır. RS mimarisi ve çözümleri üzerine, literatürden 2010-2023 yılları arasında yapılmış çeşitli çalışmalar ilgililik durumuna göre seçilmiş ve incelenmiştir. RS’lerde filtreleme teknikleri hiyerarşik olarak sınıflandırılmış ve performans değerlendirmelerinde kullanılan çoğunluk değerlendirme metrikleri saptanarak kategorize edilmiştir. Ayrıca, RS'lerden öğrenilen kazanımların İş Tavsiye Sistemleri’ndeki yansımaları araştırılmış ve IK alanındaki RS çözümleri/metrikleri ortaya konulmuştur. Son olarak, RS çözümleri üzerinde araştırma, geliştirme ve kalite değerlendirmeleri yapmak isteyen araştırmacılara, bu çalışmamız bir yol haritası niteliğindedir.

Anahtar Kelimeler

EVALUATION OF PERFORMANCE METRICS USED IN RECOMMENDATION SYSTEMS ACCORDING TO FILTERING TECHNOLOGIES: A RESEARCH STUDY ON THE FIELD OF JOB RECOMMENDATION SYSTEMS

Abstract

Thanks to Recommendation Systems (RSs), it has become possible to carry out existing processes/operations effectively in almost every sector (e.g. e-commerce, education, entertainment, healthcare, human resources, advertising, etc.) and to prioritize items that may interest the user. With the contribution of RSs, it is possible to effectively manage sectoral processes/services and produce personalized results for users. This study aims to review RS-related research, reveal a taxonomy of filtering techniques, and identify widely encountered performance metrics. In addition, Job Recommendation Systems, which are indispensable for Human Resources (HR) management, were chosen as the research area in this study and it was planned to determine performance metrics and item filtering approaches. Various studies from the literature on RS architecture and solutions, conducted between 2010 and 2023, were selected according to their relevance and reviewed. Filtering techniques in RSs are classified hierarchically and the majority evaluation metrics used in performance evaluations are determined and categorized. Additionally, the reflections of the gains learned from RSs on Job Recommendation Systems were investigated and RS solutions/metrics in the field of HR were presented. Finally, this study serves as a road map for researchers who want to conduct research, development and quality evaluations on RS solutions.

Keywords

Supporting Institution

DOĞU AKDENİZ ÜNİVERSİTESİ

Ethical Statement

Etik onay beyanı: Bu çalışma için resmi onay gerekli değildir.

References

Adomavicius, G., & Tuzhilin, A. (2011). Context-aware recommender systems. In Recommender systems handbook (pp. 217-253). Springer, Boston, MA. https://doi.org/10.1609/aimag.v32i3.2364
Al-Habaibeh, A., Watkins, M., Waried, K., & Javareshk, M. B. (2021). Challenges and opportunities of remotely working from home during Covid-19 pandemic. Global Transitions, 3, 99-108. https://doi.org/10.1016/j.glt.2021.11.001
Almalis, N. D., Tsihrintzis, G. A., Karagiannis, N., & Strati, A. D. (2015). FoDRA—A new content-based job recommendation algorithm for job seeking and recruiting. In 2015 6th International Conference on Information, Intelligence, Systems and Applications (IISA) (pp. 1-7). IEEE. https://doi.org/10.1109/IISA.2015.7388018
Al-Otaibi, S., & Ykhlef, M. (2017). Hybrid immunizing solution for job recommender system. Frontiers of Computer Science, 11(3), 511-527. https://doi.org/10.1007/s11704-016-5241-z
Al-Shamri, M. Y. H. (2016). User profiling approaches for demographic recommender systems. Knowledge-Based Systems, 100, 175-187. https://doi.org/10.1016/j.knosys.2016.03.006
Althbiti, A., Alshamrani, R., Alghamdi, T., Lee, S., & Ma, X. (2021). Addressing data sparsity in collaborative filtering-based recommender systems using clustering and artificial neural network. In 2021 IEEE 11th Annual Computing and Communication Workshop and Conference (CCWC) (pp. 0218-0227). IEEE. https://doi.org/10.1109/CCWC51732.2021.9376008
Aouadni, I., & Rebai, A. (2017). Decision support system based on genetic algorithm and multi-criteria satisfaction analysis (MUSA) method for measuring job satisfaction. Annals of Operations Research, 256(1), 3-20. https://doi.org/10.1007/s10479-016-2154-z
Arita, S., Hiyama, A., & Hirose, M. (2017). Gber: A social matching app which utilizes time, place, and skills of workers and jobs. In Companion of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (pp. 127-130). https://doi.org/10.1145/3022198.3026316

Avazpour, I., Pitakrat, T., Grunske, L., & Grundy, J. (2014). Dimensions and metrics for evaluating recommendation systems. In Recommendation systems in software engineering (pp. 245-273). Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-45135-5_10
Ayub, M., Ghazanfar, M. A., Maqsood, M., & Saleem, A. (2018). A Jaccard base similarity measure to improve performance of CF based recommender systems. In 2018 International conference on information networking (ICOIN) (pp. 1-6). IEEE. https://doi.org/10.1109/ICOIN.2018.8343073
Beel, J., Gipp, B., Langer, S., & Breitinger, C. (2016). Paper recommender systems: a literature survey. International Journal on Digital Libraries, 17(4), 305-338. https://doi.org/10.1007/s00799-015-0156-0
Benabderrahmane, S., Mellouli, N., & Lamolle, M. (2017). Predicting the users' clickstreams using time series representation, symbolic sequences, and deep learning: application on job offers recommendation tasks. In 2017 IEEE International Conference on Information Reuse and Integration (IRI) (pp. 436-443). IEEE. https://doi.org/10.1109/IRI.2017.54
Bhat, S. S., Pranav, P., Shashank, K. V., Raghunandan, A., & Mohan, B. R. (2022). Comparative Performance Evaluation of Web-Based Book Recommender Systems. In 2022 6th International Conference on Trends in Electronics and Informatics (ICOEI) (pp. 985-991). https://doi.org/IEEE. 10.1109/ICOEI53556.2022.9777116
Birjali, M., Kasri, M., & Beni-Hssane, A. (2021). A comprehensive survey on sentiment analysis: Approaches, challenges and trends. Knowledge-Based Systems, 226, 107134. ). https://doi.org/10.1016/j.knosys.2021.107134
Bobadilla, J., Ortega, F., Hernando, A., & Gutiérrez, A. (2013). Recommender systems survey. Knowledge-based systems, 46, 109-132. https://doi.org/10.1016/j.knosys.2013.03.012
Bothmer, K., & Schlippe, T. (2022). Investigating natural language processing techniques for a recommendation system to support employers, job seekers and educational institutions. In Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners’ and Doctoral Consortium: 23rd International Conference, AIED 2022, Durham, UK, July 27–31, 2022, Proceedings, Part II (pp. 449-452). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-031-11647-6_90
Burke, R., Felfernig, A., & Göker, M. H. (2011). Recommender systems: An overview. Ai Magazine, 32(3), 13-18. https://doi.org/10.1609/aimag.v32i3.2361
Çano, E., & Morisio, M. (2017). Hybrid recommender systems: A systematic literature review. Intelligent data analysis, 21(6), 1487-152. https://doi.org/10.3233/IDA-163209
Çelik Ertuğrul, D., & Elçi, A. (2020). A survey on semanticized and personalized health recommender systems. Expert Systems, 37(4), e12519. https://doi.org/10.1111/exsy.12519
Chaaya, G., Métais, E., Abdo, J. B., Chiky, R., Demerjian, J., & Barbar, K. (2017, December). Evaluating non-personalized single-heuristic active learning strategies for collaborative filtering recommender systems. In 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA) (pp. 593-600). IEEE. https://doi.org/10.1109/ICMLA.2017.00-96
Chai, Y., Wang, C., Wen, Y., & Yuan, X. (2016). A Hadoop-Based Database Querying Approach for Non-expert Users. In Asia-Pacific Web Conference (pp. 449-453). Springer, Cham.
Fang, D., Varshney, K. R., Wang, J., Ramamurthy, K. N., Mojsilovic, A., & Bauer, J. H. (2013). Quantifying and recommending expertise when new skills emerge. In 2013 IEEE 13th International Conference on Data Mining Workshops (pp. 672-679). IEEE. https://doi.org/10.1109/ICDMW.2013.33
Fkih, F. (2022). Similarity measures for Collaborative Filtering-based Recommender Systems: Review and experimental comparison. Journal of King Saud University-Computer and Information Sciences, 34(9), 7645-7669. https://doi.org/10.1016/j.jksuci.2021.09.014
Fusco, F., Vlachos, M., Vasileiadis, V., Wardatzky, K., & Schneider, J. (2019). RecoNet: An Interpretable Neural Architecture for Recommender Systems. In IJCAI (pp. 2343-2349). https://doi.org/10.24963/ijcai.2019/325
Goldberg, D., Nichols, D., Oki, B.M. and Terry, D. (1992). Using Collaborative Filtering to Weave an Information Tapestry. Communications of the ACM, 35, 61-70.http://dx.doi.org/10.1145/138859.138867.
González-Briones, A., Rivas, A., Chamoso, P., Casado-Vara, R., & Corchado, J. M. (2019). Case-based reasoning and agent-based job offer recommender system. In International Joint Conference SOCO’18-CISIS’18-ICEUTE’18: San Sebastián, Spain, June 6-8, 2018 Proceedings 13 (pp. 21-33). Springer International Publishing. https://doi.org/10.1007/978-3-319-94120-2_3
Guan, Z., Yu, B., & Liu, Y. (2019). Recruitment and Recommendation System Based on Intelligent Computing. In Proceedings of the 2019 5th International Conference on Computing and Data Engineering (pp. 77-80). https://doi.org/10.1145/3330530.3330532
Gunawardana, A., & Shani, G. (2009). A survey of accuracy evaluation metrics of recommendation tasks. Journal of Machine Learning Research, 10(12). https://doi.org/10.1145/1577069.1755883
Gunawardana, A., Shani, G., & Yogev, S. (2022). Evaluating recommender systems. In Recommender systems handbook (pp. 547-601). Springer, New York, NY. https://doi.org/10.1007/978-0-387-85820-3_8
Isinkaye, F. O., Folajimi, Y. O., & Ojokoh, B. A. (2015). Recommendation systems: Principles, methods and evaluation. Egyptian informatics journal, 16(3), 261-273. https://doi.org/10.1016/j.eij.2015.06.005
Kang, J. S., Shin, D. H., Baek, J. W., & Chung, K. (2019). Activity recommendation model using rank correlation for chronic stress management. Applied Sciences, 9(20), 4284. https://doi.org/10.3390/app9204284
Kumar, R., Verma, B. K., & Rastogi, S. S. (2014). Social popularity based SVD++ recommender system. International Journal of Computer Applications, 87(14). https://doi.org/10.5120/15279-4033
Kwieciński, R., Melniczak, G., & Górecki, T. (2023). Comparison of Real-Time and Batch Job Recommendations. IEEE Access, 11, 20553-20559. https://doi.org/10.1109/ACCESS.2023.3249356
Liang, F., & Wan, X. (2022). Job Matching Analysis Based on Text Mining and Multicriteria Decision-Making. Mathematical Problems in Engineering. https://doi.org/10.1155/2022/9245876
Liu, P., Ma, J., Wang, Y., Ma, L., & Huang, S. (2016). A context-aware method for top-k recommendation in smart TV. In Asia-Pacific Web Conference (pp. 150-161). Springer, Cham. https://doi.org/10.1007/978-3-319-45817-5_12
Lu, J., Wu, D., Mao, M., Wang, W., & Zhang, G. (2015). Recommender system application developments: a survey. Decision Support Systems, 74, 12-32. https://doi.org/10.1016/j.dss.2015.03.008
Lü, L., Medo, M., Yeung, C. H., Zhang, Y. C., Zhang, Z. K., & Zhou, T. (2012). Recommender systems. Physics reports, 519(1), 1-49. https://doi.org/10.1016/j.physrep.2012.02.006
Luo, Y., Xu, B., Cai, H., & Bu, F. (2014). A Hybrid User Profile Model for Personalized Recommender System with Linked Open Data. In 2014 Enterprise Systems Conference (pp. 243-248). IEEE. https://doi.org/10.1109/ES.2014.16
Malinowski, J., Keim, T., Wendt, O., & Weitzel, T. (2006). Matching people and jobs: A bilateral recommendation approach. In Proceedings of the 39th Annual Hawaii International Conference on System Sciences (HICSS'06) (Vol. 6, pp. 137c-137c). IEEE. https://doi.org/10.1109/HICSS.2006.266
Marlowe, A. N. (2021). Robot Recruiters: How Employers & Governments Must Confront the Discriminatory Effects of AI Hiring. J. High Tech. L., 22, 274.
Martin, F. J., Donaldson, J., Ashenfelter, A., Torrens, M., & Hangartner, R. (2011). The big promise of recommender systems. AI Magazine, 32(3), 19-27. https://doi.org/10.1609/aimag.v32i3.2360
Martinez-Gil, J., Paoletti, A. L., & Pichler, M. (2020). A novel approach for learning how to automatically match job offers and candidate profiles. Information Systems Frontiers, 22(6), 1265-1274. https://doi.org/10.1007/s10796-019-09929-7
Mathur, A., Juguru, S. K., & Eirinaki, M. (2019). A graph-based recommender system for food products. In 2019 First International Conference on Graph Computing (GC) (pp. 83-87). IEEE https://doi.org/10.1109/GC46384.2019.00020
McFee, B., & Lanckriet, G. R. (2010). Metric learning to rank. In Proceedings of the 27th international conference on machine learning (ICML-10) (pp. 775-782).
Mihalcea, R., Corley, C., & Strapparava, C. (2006). Corpus-based and knowledge-based measures of text semantic similarity. In Aaai (Vol. 6, No. 2006, pp. 775-780).
Mohamed, M. H., Khafagy, M. H., & Ibrahim, M. H. (2019). Recommender systems challenges and solutions survey. In 2019 international conference on innovative trends in computer engineering (ITCE) (pp. 149-155). IEEE. https://doi.org/10.1109/ITCE.2019.8646645
Negroponte, N. (1970). The architecture machine: toward a more human environment. The MIT Press. https://doi.org/10.7551/mitpress/8269.001.0001
Okfalisa, Siburian, R., Vitriani, Y., Rusnedy, H., Saktioto, & Yola, M. (2021). Job Training Recommendation System: Integrated Fuzzy AHP and TOPSIS Approach. In International Conference of Reliable Information and Communication Technology (pp. 84-94). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-030-98741-1_8
Painsky, A., & Wornell, G. (2018). On the universality of the logistic loss function. In 2018 IEEE International Symposium on Information Theory (ISIT) (pp. 936-940). IEEE. https://doi.org/10.48550/arXiv.1805.03804
Palomares, I., Porcel, C., Pizzato, L., Guy, I., & Herrera-Viedma, E. (2021). Reciprocal Recommender Systems: Analysis of state-of-art literature, challenges and opportunities towards social recommendation. Information Fusion, 69, 103-127. https://doi.org/10.1016/j.inffus.2020.12.001
Pan, Y., Zhang, Y., & Zhang, R. (2016). Combo-Recommendation Based on Potential Relevance of Items. In Asia-Pacific Web Conference (pp. 505-517). Springer, Cham. https://doi.org/10.1007/978-3-319-45817-5_55
Park, D. H., Kim, H. K., Choi, I. Y., & Kim, J. K. (2012). A literature review and classification of recommender systems research. Expert systems with applications, 39(11), 10059-10072. https://doi.org/10.1016/j.eswa.2012.02.038
Poriya, A., Bhagat, T., Patel, N., & Sharma, R. (2014). Non-personalized recommender systems and user-based collaborative recommender systems. Int. J. Appl. Inf. Sys, 6(9), 22-27. https://doi.org/10.5120/ijais14-451122
Portugal, I., Alencar, P., & Cowan, D. (2018). The use of machine learning algorithms in recommender systems: A systematic review. Expert Systems with Applications, 97, 205-227. https://doi.org/10.1016/j.eswa.2017.12.020
Prafajar, K. N., Vallyan, H., Candradewi, N. L. P. A., Edbert, I. S., & Suhartono, D. (2022). Multiclass job recommendation system in the IT field between classification and prediction method. In 2022 International Conference on Green Energy, Computing and Sustainable Technology (GECOST) (pp. 181-186). IEEE. https://doi.org/10.1109/GECOST55694.2022.10010659
Rimitha, S. R., Abburu, V., Kiranmai, A., Marimuthu, C., & Chandrasekaran, K. (2019). Improving Job Recommendation Using Ontological Modeling and User Profiles. In 2019 Fifteenth Int. Conference on Information Processing (ICINPRO) (pp. 1-8). IEEE. https://doi.org/10.1109/ICInPro47689.2019.9092271
Salton, G., and Buckley, C. (1997). Term weighting approaches in automatic text retrieval. In Readings in Information Retrieval. San Francisco, CA: Morgan Kaufmann Publishers. https://doi.org/10.1016/0306-4573(88)90021-0
Salton, G., and Lesk, M. (1971). Computer evaluation of indexing and text processing. Prentice Hall, Ing. Englewood Cliffs, New Jersey. 143–180. https://doi.org/10.1145/321439.321441
Salton, G., Singhal, A., Mitra, M., & Buckley, C. (1997). Automatic text structuring and summarization. Information processing & management, 33(2), 193-207. https://doi.org/10.1016/S0306-4573(96)00062-3
Schröder, G., Thiele, M., & Lehner, W. (2011). Setting goals and choosing metrics for recommender system evaluations. In UCERSTI2 workshop at the 5th ACM conference on recommender systems, Chicago, USA (Vol. 23, p. 53). https://doi.org/10.1007/s41870-018-0202-4
Selvi, C., & Sivasankar, E. (2018). A novel singularity based improved tanimoto similarity measure for effective recommendation using collaborative filtering. In 2018 8th International Conference on Cloud Computing, Data Science & Engineering (Confluence) (pp. 256-262). IEEE. https://doi.org/ 10.1109/CONFLUENCE.2018.8442697
Shakirova, E. (2017). Collaborative filtering for music recommender system. In 2017 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (EIConRus) (pp. 548-550). IEEE. https://doi.org/ 10.1109/EICONRUS.2017.7910613
Shani, G., & Gunawardana, A. (2011). Evaluating recommendation systems. In Recommender systems handbook (pp. 257-297). Springer, Boston, MA. https://doi.org/10.1007/978-0-387-85820-3_8
Shervin Minaee. (2020). 20 Popular Machine Learning Metrics. Part 2: Ranking, & Statistical Metrics, Web Site: https://towardsdatascience.com/20-popular-machine-learning-metrics-part-2-ranking-statistical-metrics-22c3e5a937b6, LVD: 13-01-2023.
Silveira, T., Zhang, M., Lin, X., Liu, Y., & Ma, S. (2019). How good your recommender system is? A survey on evaluations in recommendation. International Journal of Machine Learning and Cybernetics, 10(5), 813-831. https://doi.org/10.1007/s13042-017-0762-9
Spearman, C. (1961). The proof and measurement of association between two things, The American Journal of Psychology 15 (1904) 72–101. https://doi.org/10.1037/11491-005
Tamburri, D. A., Van Den Heuvel, W. J., & Garriga, M. (2020). Dataops for societal intelligence: a data pipeline for labor market skills extraction and matching. In 2020 IEEE 21st International Conference on Information Reuse and Integration for Data Science (IRI) (pp. 391-394). IEEE. https://doi.org/10.48550/arXiv.2104.01966
Vijaysinh Lendave. (2021). How to Measure the Success of a Recommendation System?, in Developers Corner, October 24, , Web address: https://analyticsindiamag.com/how-to-measure-the-success-of-a-recommendation-system/, LVD: 13-01-2023. https://doi.org/10.1016/j.heliyon.2023.e15108
Vrontis, D., Christofi, M., Pereira, V., Tarba, S., Makrides, A., & Trichina, E. (2022). Artificial intelligence, robotics, advanced technologies and human resource management: a systematic review. The International Journal of Human Resource Management, 33(6), 1237-1266. https://doi.org/10.1080/09585192.2020.1871398
Wenxing, H., Yiwei, C., Jianwei, Q., & Yin, H. (2015). iHR+: A mobile reciprocal job recommender system. In 2015 10th International Conference on Computer Science & Education (ICCSE) (pp. 492-495). IEEE. https://doi.org/10.1109/ICCSE.2015.7250296
Wu, S., Sun, F., Zhang, W., Xie, X., & Cui, B. (2022). Graph neural networks in recommender systems: a survey. ACM Computing Surveys, 55(5), 1-37. https://doi.org/10.1145/3535101
Xin, X., Wang, D., Ding, Y., & Lini, C. (2016). FHSM: factored hybrid similarity methods for top-n recommender systems. In Asia-Pacific Web Conference (pp. 98-110). Springer, Cham. https://doi.org/10.1007/978-3-319-45817-5_8
Yao, Y. Y. (1995). Measuring retrieval effectiveness based on user preference of documents. Journal of the American Society for Information science, 46(2), 133-145. https://doi.org/10.1002/(SICI)1097-4571(199503)46:2<133::AID-ASI6>3.0.CO;2-Z
Yi, P., Yang, C., Li, C., & Zhang, Y. (2016). A job recommendation method optimized by position descriptions and resume information. In 2016 IEEE Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC) (pp. 761-764). IEEE. https://doi.org/10.1109/IMCEC.2016.7867312
Yu, H., Liu, C., & Zhang, F. (2011). Reciprocal recommendation algorithm for the field of recruitment. Journal of Information & Computational Science, 8(16), 4061-4068.
Zhang, B., & Feng, Y. (2016). Improving temporal recommendation accuracy and diversity via long and short-term preference transfer and fusion models. In Asia-Pacific Web Conference (pp. 174-185). Springer, Cham. https://doi.org/10.1007/978-3-319-45817-5_14
Zhao, T., Wuyu, C., & Zhixiang, C. (2021). Summer Job Selection Model Based on Job Matching and Comprehensive Evaluation Algorithm. In 2021 2nd International Conference on Artificial Intelligence and Information Systems (pp. 1-5). https://doi.org/10.1145/3469213.3470394

Details

Primary Language

Turkish

Subjects

Business Process Management , Decision Support and Group Support Systems , Information Systems (Other) , Performance Evaluation

Journal Section

Research Article

Authors

Selin Bitirim
0000-0002-3575-5855
Kuzey Kıbrıs Türk Cumhuriyeti

Duygu Çelik Ertuğrul ^*
0000-0003-1380-705X
Kuzey Kıbrıs Türk Cumhuriyeti

Publication Date

September 3, 2024

Submission Date

December 27, 2023

Acceptance Date

February 17, 2024

Published in Issue

Year 2024 Volume: 27 Number: 3

DOI

https://doi.org/10.17780/ksujes.1410926

IZ

https://izlik.org/JA49JW72AH

Cite

RIS / Bibtex

APA

Bitirim, S., & Çelik Ertuğrul, D. (2024). ÖNERİ SİSTEMLERİNDE KULLANILAN PERFORMANS METRİKLERİNİN FİLTRELEME TEKNOLOJİLERİNE GÖRE DEĞERLENDİRİLMESİ: İŞ ÖNERİ SİSTEMLERİ ALANI ÜZERİNE BİR ARAŞTIRMA ÇALIŞMASI. Kahramanmaraş Sütçü İmam Üniversitesi Mühendislik Bilimleri Dergisi, 27(3), 706-725. https://doi.org/10.17780/ksujes.1410926