Login Register Cart 0
Generic placeholder image

Recent Patents on Mechanical Engineering

Editor-in-Chief

ISSN (Print): 2212-7976
ISSN (Online): 1874-477X

Back
Journal Home About Journal Editorial Board Journal Insight Current Issue Volumes/Issues
Subscribe
Research Article

Research on Foothold Optimization of the Quadruped Crawling Robot based on Reinforcement Learning

Author(s): Xiulian Liu, Peng Wang and Renquan Dong*

Volume 17, Issue 1, 2024

Published on: 06 October, 2023

Page: [11 - 22] Pages: 12

DOI: 10.2174/0122127976252847230925104722

Price: $65

conference banner
Abstract

Background: Quadruped crawling robots will be faced with stability problems when walking on a raised slope. The stability of robot is affected by gait planning and selection of its foothold in this terrain. The slope reaction force on anterior and posterior legs is uneven. The selection strategy of its foothold should achieve good performance for the stability of the quadruped crawling robot.

Objective: Aimed at the uneven problem of slope reaction force on the anterior and posterior legs of the quadruped crawling robot when walking on the raised slope, a patent method for foothold optimization using reinforcement learning based on strategy search is proposed.

Methods: The kinematic model of the quadruped crawling robot is created in D-H coordinate method. According to the gait timing sequence method, the frame description of the quadruped crawling robot's gait on the slope is proposed. The fitting polynomial coefficients and fitting curves of all joints of the leg can be obtained by using the polynomial fitting calculation method. The reinforcement learning method based on Q-learning algorithm is proposed to find the optimal foothold by interacting with the slope environment. Comparative simulation and test of other gait and climbing slope gait, the climbing slope gait with and without the Q-learning algorithm is carried out by MATLAB platform.

Results: When the quadruped crawling robot adopts the reinforcement learning method based on Qlearning algorithm to select foothold, the robot posture curves are compared without optimization strategy. The result proves that the selection strategy of its foothold is valid.

Conclusion: The selection strategy of its foothold with reinforcement learning based on Q-learning algorithm can improve the stability of the quadruped crawling robot on the raised sloped.

« Previous Next »
[1]
Shakourzadeh S, Farrokhi M. Fuzzy-backstepping control of quadruped robots. Intell Serv Robot 2020; 13(2): 191-206.
[http://dx.doi.org/10.1007/s11370-019-00309-3]
[2]
Fankhauser P. ANYmaI: A Unique Quadruped Robot Conquering Harsh Environments. Research Features 2018; 126: 54-7.
[3]
Grandia R, Farshidian F, Ranftl R. Feedback MPC for Torque-Controlled Legged Robots. IEEE Robot Autom Lett 2019; 5(15): 1-8.
[4]
Semini C, Barasuol V, Goldsmith J, et al. Design of the Hydraulically Actuated, Torque-Controlled Quadruped Robot HyQ2Max. IEEE/ASME Trans Mechatron 2017; 22(2): 635-46.
[http://dx.doi.org/10.1109/TMECH.2016.2616284]
[5]
Winkler AW, Bellicoso CD, Hutter M. ANYmaI-A Highly Mobile and Dynamic Quadrupedal Robot. 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Daejeon, Korea (South) IEEE 2016.
[6]
Hutter M, Gehring C, Lauber A, et al. ANYmal - toward legged robots for harsh environments. Adv Robot 2017; 31(17): 918-31.
[http://dx.doi.org/10.1080/01691864.2017.1378591]
[7]
Haarnoja T, Ha S, Zhou A. Learning to Walk via Deep einforcement Learning. arXiv:181211103 2019.
[8]
Park HW, Wensing PM, Kim S. High-speed bounding with the MIT Cheetah 2: Control design and experiments. Int J Robot Res 2017; 36(2): 167-92.
[http://dx.doi.org/10.1177/0278364917694244]
[9]
Ren D, Shao J, Sun G, Shao X. The Complex Dynamic Locomotive Control and Experimental Research of a Quadruped-Robot Based on the Robot Trunk. Appl Sci (Basel) 2019; 9(18): 3911.
[http://dx.doi.org/10.3390/app9183911]
[10]
Wang B, Wan Z, Zhou C, Wu J, Qiu Y, Gao Z. A Multi-module Controller for Walking Quadruped Robots. J Bionics Eng 2019; 16(2): 253-63.
[http://dx.doi.org/10.1007/s42235-019-0021-8]
[11]
Ding C, Zhou L, Rong X, Li Y, Gu J. A Lateral Impact Recovery Method for Quadruped Robot with Step Height Compensation. Int J Robot Autom 2020; 35(3): 199-208.
[http://dx.doi.org/10.2316/J.2020.206-0318]
[12]
Ding C, Zhou L, Li Y. A Novel Dynamic Locomotion Control Method for Quadruped Robots Running on Rough Terrains. IEEE Access 8: 150435-46.
[http://dx.doi.org/10.1109/ACCESS.2020.3016312]
[13]
Shi Y, Li M, Zha F, et al. Force-controlled Compensation Scheme for P-Q Valve-controlled Asymmetric Cylinder used on Hydraulic Quadruped Robots. J Bionics Eng 2020; 17(6): 1139-51.
[http://dx.doi.org/10.1007/s42235-020-0091-7]
[14]
Chen J, San H, Wu X. Gait Regulation of a Bionic Quadruped Robot with Antiparallelogram Leg Based on CPG Oscillator. Complexity 2019; 2019: 1-11.
[http://dx.doi.org/10.1155/2019/5491298]
[15]
He J, Shao J, Sun G, Shao X. Survey of Quadruped Robots Coping Strategies in Complex Situations. Electronics (Basel) 2019; 8(12): 1414.
[http://dx.doi.org/10.3390/electronics8121414]
[16]
Li T, Zhang C, Wang S, Dai JS. Jumping with Expandable Trunk of a Metamorphic Quadruped Robot—The Origaker II. Appl Sci (Basel) 2019; 9(9): 1778.
[http://dx.doi.org/10.3390/app9091778]
[17]
Guo W, Cai C, Li M, Zha F, Wang P, Jiang Z. Estimation of leg stiffness using an approximation to the planar spring–mass system in high-speed running. Int J Adv Robot Syst 2020; 17(1)
[http://dx.doi.org/10.1177/1729881419890713]
[18]
Chen T, Sun X, Xu Z, Li Y, Rong X, Zhou L. A trot and flying trot control method for quadruped robot based on optimal foot force distribution. J Bionics Eng 2019; 16(4): 621-32.
[http://dx.doi.org/10.1007/s42235-019-0050-3]
[19]
Chen J, San H, Wu X, et al. Structural Design and Gait research of a New Bionic Quadruped Robot. Proc Inst Mech Eng, B J Eng Manuf 2021; 2021: 1-13.
[20]
Neunert M, Stauble M, Giftthaler M, et al. Whole-Body Nonlinear Model Predictive Control Through Contacts for Quadrupeds. IEEE Robot Autom Lett 2018; 3(3): 1458-65.
[http://dx.doi.org/10.1109/LRA.2018.2800124]
[21]
Wang P, Song CX, Zhang Y. Three-legged robot used for road cleaning of landslides. CN Patent 2,018,204,339,013, 2018.
[22]
Wang P, Song C, Li X, Luo P. Gait planning and control of quadruped crawling robot on a slope. Ind Rob 2019; 47(1): 12-22.
[http://dx.doi.org/10.1108/IR-05-2019-0115]
[23]
Wang P, Song CX, Zhang Y, Zhang P, Li XQ, Lou P. “An emergency search and rescue quadruped robot used in landslide environment”. CN Patent 2,018,205,411,795, 2018.
[24]
Wang P, Dong R, Sun T, Tang Q. Gait Design and Analysis of Quadruped Crawling Robot Climbing Over the Raised Terrain of Slope. Recent Pat Mech Eng 2022; 15(1): 50-60.
[http://dx.doi.org/10.2174/2212797614666210413145741]
[25]
Wang P, Song CX, Zhang Y, et al. Foot structure of crawling robot suitable for slope environment. CN Patent 2,018,216,633,830, 2019.
[26]
Kumar A, Sharma R, Varshney P. Lyapunov fuzzy Markov game controller for two link robotic manipulator. J Intell Fuzzy Syst 2018; 34(3): 1479-90.
[http://dx.doi.org/10.3233/JIFS-169443]
[27]
Rao J, An H, Zhang T. Single leg operational space control of quadruped robot based on reinforcement learning. 2016 IEEE Chinese Guidance Navigation and Control Conference (CGNCC). Nanjing, China. IEEE. 2016.

Rights & Permissions Print Cite
© 2024 Bentham Science Publishers | Privacy Policy