Araştırma Makalesi
BibTex RIS Kaynak Göster

Elektrohidrolik bir sistemin pekiştirmeli öğrenme tabanlı denetleyici ile konum denetiminin gerçekleştirilmesi

Yıl 2023, Cilt: 12 Sayı: 1, 280 - 288, 15.01.2023
https://doi.org/10.28948/ngumuh.1163241

Öz

Elektrohidrolik sistemler sağladıkları avantajlar sebebiyle endüstrinin vazgeçilmezi olmuştur. Buna karşın hidrolik sistemlerin doğrusal olmayan karakteristik özellikleri ve çok sayıda parametre belirsizliği barındırması bu sistemlerin denetimini zorlaştıran etmenler olarak öne çıkmaktadır. Bu çalışmada ise oransal valf ile sürülen asimetrik bir hidrolik pistonun konumu pekiştirmeli öğrenme ile denetlenmiştir. Pek çok pekiştirmeli öğrenme algoritması olmasına rağmen sürekli uzayda etkinliği ile öne çıkan derin deterministik politika gradyanı yöntemi tercih edilmiştir. İlgili hiper parametreler öncül çalışmalarla belirlenerek çoklu konum referans sinyali için denetleyicinin eğitimi benzetim ortamında gerçekleştirilmiştir. Elde edilen sonuçları kıyaslamak için aynı çalışma PID denetleyici ile de gerçekleştirilmiştir. Çalışmada kullanılan pekiştirmeli öğrenme yöntemi farklı karakteristiklere sahip konum referans sinyalinin takibinde PID denetleyiciden daha %25.51 oranında daha başarılı sonuçlar ortaya koymuştur.

Kaynakça

  • J. Činkelj, R. Kamnik, P. Čepon, M. Mihelj and M. Munih, Closed-loop control of hydraulic telescopic handler. Automation in Construction, 19, 954–963, 2010. https://doi.org/10.1016/j.autcon.2010.07.012
  • M. Borghi, B. Zardin, F. Pintore and F. Belluzzi, Energy savings in the hydraulic circuit of agricultural tractors. Energy Procedia, Elsevier B.V. 45, 352–361, 2014. https://doi.org/10.1016/j.egypro.2014.01.038
  • T. Boaventura, J. Buchli, C. Semini and D.G. Caldwell, Model-Based hydraulic impedance control for dynamic robots. IEEE Transactions on Robotics, IEEE. 31, 1324–1336, 2015. https://doi.org/10.1109/ TRO.2015.2482061
  • A. Bayrak, E. Tatlicioglu and E. Zergeroglu, Backstepping control of electro-hydraulic arm. 2018 6th International Conference on Control Engineering and Information Technology, CEIT 2018, IEEE. pp. 1-4, 2018. https://doi.org/10.1109/CEIT.2018.8751833
  • D. Janosevic, R. Mitrev, B. Andjelkovic, and P. Petrov, Quantitative measures for assessment of the hydraulic excavator digging efficiency. Journal of Zhejiang University: Science A, 13, 926–942, 2012. https://doi.org/10.1631/jzus.A1100318
  • M. Heikkilä and M. Linjama, Displacement control of a mobile crane using a digital hydraulic power management system. Mechatronics, 23, 452–461, 2013. https://doi.org/10.1016/j.mechatronics.2013.03. 009
  • M. Jelali and A. Kroll, Hydraulic Servo-systems. Springer London, London. 2003. https://doi.org/ 10.1007/978-1-4471-0099-7
  • M. Fallahi, M. Zareinejad, K. Baghestan, A. Tivay, S.M. Rezaei and A. Abdullah, Precise position control of an electro-hydraulic servo system via robust linear approximation. ISA Transactions, Elsevier Ltd. 80, 503–512, 2018. https://doi.org/10.1016/j.isatra. 2018.06.002
  • C. Onat, M. Daskin and A. Turan, Gain scheduling linear model of an electro-hydraulic actuator. An International Journal of Optimization and Control: Theories & Applications (IJOCTA), 7, 301–306, 2017. https://doi.org/10.11121/ijocta.01.2017.00487
  • E. Kolsi-Gdoura, M. Feki and N. Derbel, Sliding mode-based robust position control of an electrohydraulic system. 10th International Multi-Conferences on Systems, Signals & Devices 2013 (SSD13), IEEE. pp. 1–5 2013. https://doi.org/10.1109/SSD.2013.6564127
  • E. Kolsi Gdoura, M. Feki and N. Derbel, Sliding mode control of a hydraulic servo system position using adaptive sliding surface and adaptive gain. International Journal of Modelling, Identification and Control, 23, 248–259, 2015. https://doi.org/10.1504/ IJMIC.2015.069946
  • B. Shaer, J.-P. Kenné, C. Kaddissi and C. Fallaha, A chattering-free fuzzy hybrid sliding mode control of an electrohydraulic active suspension. Transactions of the Institute of Measurement and Control, 40, 222–238, 2018. https://doi.org/10.1177/0142331216652468
  • M. Kalyoncu and M. Haydim, Mathematical modelling and fuzzy logic based position control of an electrohydraulic servosystem with internal leakage. Mechatronics, Elsevier Ltd. 19, 847–858, 2009. https://doi.org/10.1016/j.mechatronics.2009.04.010
  • L. Yu, L. Ding, F. Yu, J. Zheng and Y. Tian, Force tracking control for electrohydraulic servo system based on adaptive neuro-fuzzy inference system (ANFIS) controller. International Journal of Intelligent Computing and Cybernetics, 14, 1–16, 2021. https://doi.org/10.1108/IJICC-09-2020-0132
  • Y.J. Liu, Y.D. Xie and H. Wang, Fuzzy PID control for valve-controlled cylinder hydraulic system. Applied Mechanics and Materials, 212–213, 1244–1248, 2012. https://doi.org/10.4028/www.scientific.net/AMM.212-213.1244
  • B.K. Sarkar, P. Mandal, R. Saha, S. Mookherjee and D. Sanyal, GA-optimized feedforward-PID tracking control for a rugged electrohydraulic system design. ISA Transactions, Elsevier. 52, 853–861, 2013. https://doi.org/10.1016/j.isatra.2013.07.008
  • A. Rodriguez-Ramos, C. Sampedro, H. Bavle, P. de la Puente and P. Campoy, A Deep Reinforcement Learning Strategy for UAV Autonomous Landing on a Moving Platform. Journal of Intelligent & Robotic Systems, Journal of Intelligent & Robotic Systems. 93, 351–366, 2019. https://doi.org/10.1007/s10846-018-0891-8
  • X. Wu, S. Liu, T. Zhang, L. Yang, Y. Li and T. Wang, Motion Control for Biped Robot via DDPG-based Deep Reinforcement Learning. 2018 WRC Symposium on Advanced Robotics and Automation, WRC SARA 2018 - Proceeding, IEEE. 40–45, 2018. https://doi.org/10.1109/WRC-SARA.2018.8584227
  • Y. Hou, L. Liu, Q. Wei, X. Xu and C. Chen, A novel DDPG method with prioritized experience replay. 2017 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2017, 2017-Janua, pp. 316–321, 2017. https://doi.org/10.1109/SMC.2017.8122622
  • P. Wos and R. Dindorf, Modeling and Analysis of the Hydraulic Servo Drive System. Advances in Intelligent Systems and Computing, Springer International Publishing, 253-262 2015. https://doi.org/10.1007/978-3-319-26886-6
  • L. Lu and B. Yao, Energy-saving adaptive robust control of a hydraulic manipulator using five cartridge valves with an accumulator. IEEE Transactions on Industrial Electronics, 61, 7046–7054, 2014. https://doi.org/10.1109/TIE.2014.2314054
  • U. Pinsopon, T. Hwang, S. Cetinkunt, R. Ingram, Q. Zhang, M. Cobo, D. Koehler and R. Ottman, Hydraulic actuator control with open-centre electrohydraulic valve using a cerebellar model articulation controller neural network algorithm. Proceedings of the Institution of Mechanical Engineers Part I: Journal of Systems and Control Engineering, 213, 33–48, 1999. https://doi.org/10.1243/0959651991540368
  • S. Armoogum and X. Li, Big Data Analytics and Deep Learning in Bioinformatics With Hadoop. Deep Learning and Parallel Computing Environment for Bioengineering Systems, Elsevier. pp. 17–36 2019. https://doi.org/10.1016/B978-0-12-816718-2.00009-9
  • R.S. Sutton and A.G. Barto, Reinforcement Learning: An Introduction. The MIT Press, London, 2018.
  • T.P. Lillicrap, J.J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver and D. Wierstra, Continuous control with deep reinforcement learning. 2015. https://doi.org/10.48550/arXiv.1509.02971
  • S. Guo, X. Zhang, Y. Zheng and Y. Du, An autonomous path planning model for unmanned ships based on deep reinforcement learning. Sensors (Switzerland), 20, 2020. https://doi.org/10.3390/s20020426

Reinforcement learning based position control of an electro-hydraulic system

Yıl 2023, Cilt: 12 Sayı: 1, 280 - 288, 15.01.2023
https://doi.org/10.28948/ngumuh.1163241

Öz

Electrohydraulic systems have become an inseparable part of the industry due to the advantages they provide. On the other hand, the nonlinear characteristics of the hydraulic systems and the parametric uncertainties make their control troublesome. In this study, the position of an asymmetrical hydraulic piston driven by a proportional valve was controlled by reinforcement learning. Although there are many reinforcement learning algorithms, the deep deterministic policy gradient, which stands out with its effectiveness in continuous space, has been preferred. The hyperparameters were found by preliminary studies and the training of the controller for the designed position reference signal was carried out numerically. The obtained results are compared with the PID controller. The reinforcement learning method reached 25.51% more successful results than the PID controller in terms of tracking the position reference signal with different characteristics.

Kaynakça

  • J. Činkelj, R. Kamnik, P. Čepon, M. Mihelj and M. Munih, Closed-loop control of hydraulic telescopic handler. Automation in Construction, 19, 954–963, 2010. https://doi.org/10.1016/j.autcon.2010.07.012
  • M. Borghi, B. Zardin, F. Pintore and F. Belluzzi, Energy savings in the hydraulic circuit of agricultural tractors. Energy Procedia, Elsevier B.V. 45, 352–361, 2014. https://doi.org/10.1016/j.egypro.2014.01.038
  • T. Boaventura, J. Buchli, C. Semini and D.G. Caldwell, Model-Based hydraulic impedance control for dynamic robots. IEEE Transactions on Robotics, IEEE. 31, 1324–1336, 2015. https://doi.org/10.1109/ TRO.2015.2482061
  • A. Bayrak, E. Tatlicioglu and E. Zergeroglu, Backstepping control of electro-hydraulic arm. 2018 6th International Conference on Control Engineering and Information Technology, CEIT 2018, IEEE. pp. 1-4, 2018. https://doi.org/10.1109/CEIT.2018.8751833
  • D. Janosevic, R. Mitrev, B. Andjelkovic, and P. Petrov, Quantitative measures for assessment of the hydraulic excavator digging efficiency. Journal of Zhejiang University: Science A, 13, 926–942, 2012. https://doi.org/10.1631/jzus.A1100318
  • M. Heikkilä and M. Linjama, Displacement control of a mobile crane using a digital hydraulic power management system. Mechatronics, 23, 452–461, 2013. https://doi.org/10.1016/j.mechatronics.2013.03. 009
  • M. Jelali and A. Kroll, Hydraulic Servo-systems. Springer London, London. 2003. https://doi.org/ 10.1007/978-1-4471-0099-7
  • M. Fallahi, M. Zareinejad, K. Baghestan, A. Tivay, S.M. Rezaei and A. Abdullah, Precise position control of an electro-hydraulic servo system via robust linear approximation. ISA Transactions, Elsevier Ltd. 80, 503–512, 2018. https://doi.org/10.1016/j.isatra. 2018.06.002
  • C. Onat, M. Daskin and A. Turan, Gain scheduling linear model of an electro-hydraulic actuator. An International Journal of Optimization and Control: Theories & Applications (IJOCTA), 7, 301–306, 2017. https://doi.org/10.11121/ijocta.01.2017.00487
  • E. Kolsi-Gdoura, M. Feki and N. Derbel, Sliding mode-based robust position control of an electrohydraulic system. 10th International Multi-Conferences on Systems, Signals & Devices 2013 (SSD13), IEEE. pp. 1–5 2013. https://doi.org/10.1109/SSD.2013.6564127
  • E. Kolsi Gdoura, M. Feki and N. Derbel, Sliding mode control of a hydraulic servo system position using adaptive sliding surface and adaptive gain. International Journal of Modelling, Identification and Control, 23, 248–259, 2015. https://doi.org/10.1504/ IJMIC.2015.069946
  • B. Shaer, J.-P. Kenné, C. Kaddissi and C. Fallaha, A chattering-free fuzzy hybrid sliding mode control of an electrohydraulic active suspension. Transactions of the Institute of Measurement and Control, 40, 222–238, 2018. https://doi.org/10.1177/0142331216652468
  • M. Kalyoncu and M. Haydim, Mathematical modelling and fuzzy logic based position control of an electrohydraulic servosystem with internal leakage. Mechatronics, Elsevier Ltd. 19, 847–858, 2009. https://doi.org/10.1016/j.mechatronics.2009.04.010
  • L. Yu, L. Ding, F. Yu, J. Zheng and Y. Tian, Force tracking control for electrohydraulic servo system based on adaptive neuro-fuzzy inference system (ANFIS) controller. International Journal of Intelligent Computing and Cybernetics, 14, 1–16, 2021. https://doi.org/10.1108/IJICC-09-2020-0132
  • Y.J. Liu, Y.D. Xie and H. Wang, Fuzzy PID control for valve-controlled cylinder hydraulic system. Applied Mechanics and Materials, 212–213, 1244–1248, 2012. https://doi.org/10.4028/www.scientific.net/AMM.212-213.1244
  • B.K. Sarkar, P. Mandal, R. Saha, S. Mookherjee and D. Sanyal, GA-optimized feedforward-PID tracking control for a rugged electrohydraulic system design. ISA Transactions, Elsevier. 52, 853–861, 2013. https://doi.org/10.1016/j.isatra.2013.07.008
  • A. Rodriguez-Ramos, C. Sampedro, H. Bavle, P. de la Puente and P. Campoy, A Deep Reinforcement Learning Strategy for UAV Autonomous Landing on a Moving Platform. Journal of Intelligent & Robotic Systems, Journal of Intelligent & Robotic Systems. 93, 351–366, 2019. https://doi.org/10.1007/s10846-018-0891-8
  • X. Wu, S. Liu, T. Zhang, L. Yang, Y. Li and T. Wang, Motion Control for Biped Robot via DDPG-based Deep Reinforcement Learning. 2018 WRC Symposium on Advanced Robotics and Automation, WRC SARA 2018 - Proceeding, IEEE. 40–45, 2018. https://doi.org/10.1109/WRC-SARA.2018.8584227
  • Y. Hou, L. Liu, Q. Wei, X. Xu and C. Chen, A novel DDPG method with prioritized experience replay. 2017 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2017, 2017-Janua, pp. 316–321, 2017. https://doi.org/10.1109/SMC.2017.8122622
  • P. Wos and R. Dindorf, Modeling and Analysis of the Hydraulic Servo Drive System. Advances in Intelligent Systems and Computing, Springer International Publishing, 253-262 2015. https://doi.org/10.1007/978-3-319-26886-6
  • L. Lu and B. Yao, Energy-saving adaptive robust control of a hydraulic manipulator using five cartridge valves with an accumulator. IEEE Transactions on Industrial Electronics, 61, 7046–7054, 2014. https://doi.org/10.1109/TIE.2014.2314054
  • U. Pinsopon, T. Hwang, S. Cetinkunt, R. Ingram, Q. Zhang, M. Cobo, D. Koehler and R. Ottman, Hydraulic actuator control with open-centre electrohydraulic valve using a cerebellar model articulation controller neural network algorithm. Proceedings of the Institution of Mechanical Engineers Part I: Journal of Systems and Control Engineering, 213, 33–48, 1999. https://doi.org/10.1243/0959651991540368
  • S. Armoogum and X. Li, Big Data Analytics and Deep Learning in Bioinformatics With Hadoop. Deep Learning and Parallel Computing Environment for Bioengineering Systems, Elsevier. pp. 17–36 2019. https://doi.org/10.1016/B978-0-12-816718-2.00009-9
  • R.S. Sutton and A.G. Barto, Reinforcement Learning: An Introduction. The MIT Press, London, 2018.
  • T.P. Lillicrap, J.J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver and D. Wierstra, Continuous control with deep reinforcement learning. 2015. https://doi.org/10.48550/arXiv.1509.02971
  • S. Guo, X. Zhang, Y. Zheng and Y. Du, An autonomous path planning model for unmanned ships based on deep reinforcement learning. Sensors (Switzerland), 20, 2020. https://doi.org/10.3390/s20020426
Toplam 26 adet kaynakça vardır.

Ayrıntılar

Birincil Dil Türkçe
Konular Makine Mühendisliği
Bölüm Makine Mühendisliği
Yazarlar

Mustafa Yavuz Coşkun 0000-0002-6461-076X

Mehmet İtik 0000-0002-3827-5697

Yayımlanma Tarihi 15 Ocak 2023
Gönderilme Tarihi 17 Ağustos 2022
Kabul Tarihi 9 Ekim 2022
Yayımlandığı Sayı Yıl 2023 Cilt: 12 Sayı: 1

Kaynak Göster

APA Coşkun, M. Y., & İtik, M. (2023). Elektrohidrolik bir sistemin pekiştirmeli öğrenme tabanlı denetleyici ile konum denetiminin gerçekleştirilmesi. Niğde Ömer Halisdemir Üniversitesi Mühendislik Bilimleri Dergisi, 12(1), 280-288. https://doi.org/10.28948/ngumuh.1163241
AMA Coşkun MY, İtik M. Elektrohidrolik bir sistemin pekiştirmeli öğrenme tabanlı denetleyici ile konum denetiminin gerçekleştirilmesi. NÖHÜ Müh. Bilim. Derg. Ocak 2023;12(1):280-288. doi:10.28948/ngumuh.1163241
Chicago Coşkun, Mustafa Yavuz, ve Mehmet İtik. “Elektrohidrolik Bir Sistemin pekiştirmeli öğrenme Tabanlı Denetleyici Ile Konum Denetiminin gerçekleştirilmesi”. Niğde Ömer Halisdemir Üniversitesi Mühendislik Bilimleri Dergisi 12, sy. 1 (Ocak 2023): 280-88. https://doi.org/10.28948/ngumuh.1163241.
EndNote Coşkun MY, İtik M (01 Ocak 2023) Elektrohidrolik bir sistemin pekiştirmeli öğrenme tabanlı denetleyici ile konum denetiminin gerçekleştirilmesi. Niğde Ömer Halisdemir Üniversitesi Mühendislik Bilimleri Dergisi 12 1 280–288.
IEEE M. Y. Coşkun ve M. İtik, “Elektrohidrolik bir sistemin pekiştirmeli öğrenme tabanlı denetleyici ile konum denetiminin gerçekleştirilmesi”, NÖHÜ Müh. Bilim. Derg., c. 12, sy. 1, ss. 280–288, 2023, doi: 10.28948/ngumuh.1163241.
ISNAD Coşkun, Mustafa Yavuz - İtik, Mehmet. “Elektrohidrolik Bir Sistemin pekiştirmeli öğrenme Tabanlı Denetleyici Ile Konum Denetiminin gerçekleştirilmesi”. Niğde Ömer Halisdemir Üniversitesi Mühendislik Bilimleri Dergisi 12/1 (Ocak 2023), 280-288. https://doi.org/10.28948/ngumuh.1163241.
JAMA Coşkun MY, İtik M. Elektrohidrolik bir sistemin pekiştirmeli öğrenme tabanlı denetleyici ile konum denetiminin gerçekleştirilmesi. NÖHÜ Müh. Bilim. Derg. 2023;12:280–288.
MLA Coşkun, Mustafa Yavuz ve Mehmet İtik. “Elektrohidrolik Bir Sistemin pekiştirmeli öğrenme Tabanlı Denetleyici Ile Konum Denetiminin gerçekleştirilmesi”. Niğde Ömer Halisdemir Üniversitesi Mühendislik Bilimleri Dergisi, c. 12, sy. 1, 2023, ss. 280-8, doi:10.28948/ngumuh.1163241.
Vancouver Coşkun MY, İtik M. Elektrohidrolik bir sistemin pekiştirmeli öğrenme tabanlı denetleyici ile konum denetiminin gerçekleştirilmesi. NÖHÜ Müh. Bilim. Derg. 2023;12(1):280-8.

download