GENETİK ALGORİTMA DESTEKLİ BİR YAKLAŞIM İLE DERİN ÖĞRENME PERFORMANSININ GELİŞTİRİLMESİ: LSTM ODAKLI
Abstract
Derin öğrenme, görüntü sınıflandırma, doğal dil işleme ve konuşma tanıma gibi çeşitli uygulamalarda dikkat çekici başarılar elde etmiştir. Ancak, derin sinir ağlarını eğitmek, karmaşık mimarileri ve gereken parametre sayısı nedeniyle zorlu bir süreçtir. Genetik algoritmalar, derin öğrenme için alternatif bir optimizasyon teknik olarak önerilmiştir ve optimal bir ağ parametre setini minimize eden bir amaç fonksiyonu bulmak için etkili bir alternatif yöntem sunar. Bu makalede, derin öğrenme ile genetik algoritmaları entegre eden, özellikle LSTM modellerini kullanarak performansı artırmayı amaçlayan yeni bir yaklaşım öneriyoruz. Yöntemimiz, genetik algoritmalar aracılığıyla öğrenme hızı, grup boyutu, katman başına nöron sayısı ve katman derinliği gibi kritik hiper-parametreleri optimize eder. Ayrıca, genetik algoritma parametrelerinin optimizasyon sürecini nasıl etkilediğine dair kapsamlı bir analiz yaparak, LSTM model performansını iyileştirmedeki önemli etkilerini gösteriyoruz. Genel olarak, sunulan yöntem, derin sinir ağlarının performansını artırmak için güçlü bir mekanizma sunmakta olup bu nedenle yapay zekâ disiplininde gelecekteki uygulamalar için önemli bir potansiyele sahip olduğuna inanıyoruz.
References
- Bozkurt, B., Coskun, K., & Bakal, G. (2024). Building a challenging medical dataset for comparative evaluation of classifier capabilities. Computers in Biology and Medicine, 178, 108721.
- Chollet, F., et al. (2015). Keras. https://keras.io.
- Fang, Y., & Li, J. (2010). A review of tournament selection in genetic programming. Advances in computation and intelligence: 5th international symposium, isica 2010, wuhan, china, october 22-24, 2010. proceedings 5 (pp. 181–192).
- Greenstein, B.L., Elsey, D.C., Hutchison, G.R. (2023). Determining best practices for using genetic algorithms in molecular discovery. The Journal of Chemical Physics, 159 (9).
- Hajireza, M., Darabi, R., Najafi Moghaddam, A. (2023). The impact of accruals and free cash flow on financial stability using genetic algorithm. Accounting and Auditing Studies.
- Haldurai, L., Madhubala, T., Rajalakshmi, R. (2016). A study on genetic algorithm and its applications. Int. J. Comput. Sci. Eng, 4 (10), 139–143.
- Holland, J.H. (1992a). Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. MIT Press.
- Holland, J.H. (1992b). Genetic algorithms. Scientific American, 267 (1), 66–73, Retrieved 2023-05-07, from http://www.jstor.org/stable/24939139.
- Jebari, K. (2013, 12). Selection methods for genetic algorithms. International Journal of Emerging Sciences, 3, 333-344.
- Kallumadi, S., & Grer, F. (2018). Drug Review Dataset (Drugs.com). UCI Machine Learning Repository. (DOI: https://doi.org/10.24432/C5SK5S).
- Katoch, S., Chauhan, S.S., Kumar, V. (2021). A review on genetic algorithm: past, present, and future. Multimedia Tools and Applications, 80 , 8091–8126.
- Kolukisa, B., Dedeturk, B.K., Dedeturk, B.A., Gulsen, A., Bakal, G. (2021). A comparative analysis on medical article classification using text mining & machine learning algorithms. 2021 6th International Conference on Computer Science and engineering (UBMK) (p. 360-365).
- Kora, P., & Yadlapalli, P. (2017, 03). Crossover operators in genetic algorithms: A review. International Journal of Computer Applications, 162, 34-36, https:// doi.org/10.5120/ijca2017913370. Kramer, O., & Kramer, O. (2017). Genetic algorithms. Springer.
- Lambora, A., Gupta, K., Chopra, K. (2019). Genetic algorithm-a literature review. 2019 international conference on machine learning, big data, cloud and parallel computing (comitcon) (pp. 380–384).
- Lipowski, A., & Lipowska, D. (2012). Roulette-wheel selection via stochastic acceptance. Physica A: Statistical Mechanics and its Applications, 391 (6), 2193–2196.
- Pachuau, J.L., Roy, A., Kumar Saha, A. (2021). An overview of crossover techniques in genetic algorithm. Modeling, Simulation and Optimization: Proceedings of CoMSO 2020, 581–598.
- Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., . . .others (2011). Scikit-learn: Machine learning in python. the Journal of machine Learning research, 12, 2825–2830.
- Pencheva, T., Atanassov, K., Shannon, A. (2009). Modelling of a stochastic universal sampling selection operator in genetic algorithms using generalized nets. Proceedings of the tenth international workshop on generalized nets, sofia (pp. 1–7).
- Peng, K., Du, J., Lu, F., Sun, Q., Dong, Y., Zhou, P., Hu, M. (2019). A hybrid genetic algorithm on routing and scheduling for vehicle-assisted multi-drone parcel delivery. IEEE Access, 7, 49191–49200.
- Rathore, H., & Rathore, H. (2016). Genetic algorithms. Mapping Biological Systems to Network Systems, 97–106. Sherstinsky, A. (2020). Fundamentals of recurrent neural network (rnn) and long short-term memory (lstm) network. Physica D: Nonlinear Phenomena, 404, 132306.
- Yan, J. (2023). Adaptive scheduling of agricultural machinery equipment production lines for intelligent manufacturing. International Journal of Manufacturing Technology and Management, 37 (3/4), 349–361.
- Zheng, L., & Wen, Y. (2023). A multi-strategy differential evolution algorithm with adaptive similarity selection rule. Symmetry, 15 (9), 1697.
- Zivkovic, M., K, V., Bacanin, N., Djordjevic, A., Antonijevic, M., Strumberger, I., Rashid, T.A. (2021). Hybrid genetic algorithm and machine learning method for covid-19 cases prediction. Proceedings of international conference on sustainable expert systems: Icses 2020 (pp. 169–184).
- Şen, T. Ü., & Bakal, G. (2023). A transfer learning application on the reliability of psychological drugs’ comments. 2023 international conference on smart applications, communications and networking (smartnets) (p. 1-6).
Abstract
Deep learning has shown remarkable success in various applications, such as image classification, natural language processing, and speech recognition. However, training deep neural networks is challenging due to their complex architecture and the number of parameters required. Genetic algorithms have been proposed as an alternative optimization technique for deep learning, offering an efficient alternative way to find an optimal set of network parameters that minimize the objective function. In this paper, we propose a novel approach integrating genetic algorithms with deep learning, specifically LSTM models, to enhance performance. Our method optimizes crucial hyper-parameters including learning rate, batch size, neuron count per layer, and layer depth through genetic algorithms. Additionally, we conduct a comprehensive analysis of how genetic algorithm parameters influence the optimization process and illustrate their significant impact on improving LSTM model performance. Overall, the presented method provides a powerful mechanism for improving the performance of deep neural networks, and; thus, we believe that it has significant potential for future applications in the artificial intelligence discipline.
References
- Bozkurt, B., Coskun, K., & Bakal, G. (2024). Building a challenging medical dataset for comparative evaluation of classifier capabilities. Computers in Biology and Medicine, 178, 108721.
- Chollet, F., et al. (2015). Keras. https://keras.io.
- Fang, Y., & Li, J. (2010). A review of tournament selection in genetic programming. Advances in computation and intelligence: 5th international symposium, isica 2010, wuhan, china, october 22-24, 2010. proceedings 5 (pp. 181–192).
- Greenstein, B.L., Elsey, D.C., Hutchison, G.R. (2023). Determining best practices for using genetic algorithms in molecular discovery. The Journal of Chemical Physics, 159 (9).
- Hajireza, M., Darabi, R., Najafi Moghaddam, A. (2023). The impact of accruals and free cash flow on financial stability using genetic algorithm. Accounting and Auditing Studies.
- Haldurai, L., Madhubala, T., Rajalakshmi, R. (2016). A study on genetic algorithm and its applications. Int. J. Comput. Sci. Eng, 4 (10), 139–143.
- Holland, J.H. (1992a). Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. MIT Press.
- Holland, J.H. (1992b). Genetic algorithms. Scientific American, 267 (1), 66–73, Retrieved 2023-05-07, from http://www.jstor.org/stable/24939139.
- Jebari, K. (2013, 12). Selection methods for genetic algorithms. International Journal of Emerging Sciences, 3, 333-344.
- Kallumadi, S., & Grer, F. (2018). Drug Review Dataset (Drugs.com). UCI Machine Learning Repository. (DOI: https://doi.org/10.24432/C5SK5S).
- Katoch, S., Chauhan, S.S., Kumar, V. (2021). A review on genetic algorithm: past, present, and future. Multimedia Tools and Applications, 80 , 8091–8126.
- Kolukisa, B., Dedeturk, B.K., Dedeturk, B.A., Gulsen, A., Bakal, G. (2021). A comparative analysis on medical article classification using text mining & machine learning algorithms. 2021 6th International Conference on Computer Science and engineering (UBMK) (p. 360-365).
- Kora, P., & Yadlapalli, P. (2017, 03). Crossover operators in genetic algorithms: A review. International Journal of Computer Applications, 162, 34-36, https:// doi.org/10.5120/ijca2017913370. Kramer, O., & Kramer, O. (2017). Genetic algorithms. Springer.
- Lambora, A., Gupta, K., Chopra, K. (2019). Genetic algorithm-a literature review. 2019 international conference on machine learning, big data, cloud and parallel computing (comitcon) (pp. 380–384).
- Lipowski, A., & Lipowska, D. (2012). Roulette-wheel selection via stochastic acceptance. Physica A: Statistical Mechanics and its Applications, 391 (6), 2193–2196.
- Pachuau, J.L., Roy, A., Kumar Saha, A. (2021). An overview of crossover techniques in genetic algorithm. Modeling, Simulation and Optimization: Proceedings of CoMSO 2020, 581–598.
- Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., . . .others (2011). Scikit-learn: Machine learning in python. the Journal of machine Learning research, 12, 2825–2830.
- Pencheva, T., Atanassov, K., Shannon, A. (2009). Modelling of a stochastic universal sampling selection operator in genetic algorithms using generalized nets. Proceedings of the tenth international workshop on generalized nets, sofia (pp. 1–7).
- Peng, K., Du, J., Lu, F., Sun, Q., Dong, Y., Zhou, P., Hu, M. (2019). A hybrid genetic algorithm on routing and scheduling for vehicle-assisted multi-drone parcel delivery. IEEE Access, 7, 49191–49200.
- Rathore, H., & Rathore, H. (2016). Genetic algorithms. Mapping Biological Systems to Network Systems, 97–106. Sherstinsky, A. (2020). Fundamentals of recurrent neural network (rnn) and long short-term memory (lstm) network. Physica D: Nonlinear Phenomena, 404, 132306.
- Yan, J. (2023). Adaptive scheduling of agricultural machinery equipment production lines for intelligent manufacturing. International Journal of Manufacturing Technology and Management, 37 (3/4), 349–361.
- Zheng, L., & Wen, Y. (2023). A multi-strategy differential evolution algorithm with adaptive similarity selection rule. Symmetry, 15 (9), 1697.
- Zivkovic, M., K, V., Bacanin, N., Djordjevic, A., Antonijevic, M., Strumberger, I., Rashid, T.A. (2021). Hybrid genetic algorithm and machine learning method for covid-19 cases prediction. Proceedings of international conference on sustainable expert systems: Icses 2020 (pp. 169–184).
- Şen, T. Ü., & Bakal, G. (2023). A transfer learning application on the reliability of psychological drugs’ comments. 2023 international conference on smart applications, communications and networking (smartnets) (p. 1-6).
Details
| Primary Language | English |
|---|---|
| Subjects | Deep Learning, Neural Networks, Evolutionary Computation |
| Journal Section | Computer Engineering |
| Authors | |
| Publication Date | December 3, 2024 |
| Submission Date | April 29, 2024 |
| Acceptance Date | July 25, 2024 |
| Published in Issue | Year 2024 |