{"id":16384,"date":"2024-05-09T10:28:34","date_gmt":"2024-05-09T08:28:34","guid":{"rendered":"https:\/\/tech4future.info\/?p=16384"},"modified":"2024-12-12T11:00:36","modified_gmt":"2024-12-12T10:00:36","slug":"humanoid-robot-locomotion-natural-terrain","status":"publish","type":"post","link":"https:\/\/tech4future.info\/en\/humanoid-robot-locomotion-natural-terrain\/","title":{"rendered":"The role of reinforcement learning in controlling humanoid robots\u2019 locomotion on natural terrain"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In the December 2017 issue of Science Robotics, within the editorial &#8220;<a href=\"https:\/\/www.science.org\/doi\/10.1126\/scirobotics.aar4043\" target=\"_blank\" rel=\"noreferrer noopener\">Humanoid robotics &#8211; History, current state of the art, and challenges<\/a>&#8220;, Toshio Fukuda, Professor of Intelligent Robotics and AI at Nagoya University in Japan, Paolo Dario, then Director of the Institute of BioRobotics at the Scuola Superiore Sant&#8217;Anna in Pisa, and Guang-Zhong Yang, Director of the Hamlyn Center for Robotic Surgery at Imperial College London, characterised&nbsp;<strong>humanoid robots<\/strong>&nbsp;as &#8220;<strong>inherently interdisciplinary<\/strong>&#8220;.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u00ab<em>They considered them emblematic of one of the ultimate objectives of robotics<\/em>\u00bb: the integration of all advancements from the diverse disciplines engaged in the research and development activities focused on these anthropomorphic machines. The authors underscored fields such as&nbsp;\u00ab<em>advanced locomotion and manipulation, biomechanics, artificial intelligence, computer vision,&nbsp;<a href=\"https:\/\/tech4future.info\/en\/visual-perception-in-service-robots\/\" target=\"_blank\" rel=\"noreferrer noopener\">perceptual studies<\/a>, behavioural research, and cognitive learning<\/em>\u00bb.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Among these disciplines, the challenge of&nbsp;<strong>autonomous locomotion in complex external environments<\/strong>&nbsp;&#8211;&nbsp;\u00ab<em>beyond the confined spaces of laboratories<\/em>\u00bb&nbsp;&#8211; was identified nearly six and a half years ago as a domain still replete&nbsp;\u00ab<em>with unresolved research issues concerning humanoid robots<\/em>\u00bb.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Despite numerous global academic endeavours to address these challenges, problems related to&nbsp;<strong>stability and dynamic control of&nbsp;humanoid robot locomotion<\/strong>&nbsp;<strong>across natural terrains&nbsp;<\/strong>persist.<\/p>\n\n\n\n<div style=\"height:15px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading has-text-color has-link-color wp-elements-1\" id=\"h-t-akeaways\" style=\"color:#73cc49\"><strong>T<\/strong>AKEAWAYS<\/h2>\n\n\n\n<div class=\"wp-block-uagb-icon-list uagb-block-d48856cb\"><div class=\"uagb-icon-list__wrap\">\n<div class=\"wp-block-uagb-icon-list-child uagb-block-58750049\"><span class=\"uagb-icon-list__source-wrap\"><svg xmlns=\"https:\/\/www.w3.org\/2000\/svg\" viewBox=\"0 0 512 512\"><path d=\"M256 0C114.6 0 0 114.6 0 256c0 141.4 114.6 256 256 256s256-114.6 256-256C512 114.6 397.4 0 256 0zM406.6 278.6l-103.1 103.1c-12.5 12.5-32.75 12.5-45.25 0s-12.5-32.75 0-45.25L306.8 288H128C110.3 288 96 273.7 96 256s14.31-32 32-32h178.8l-49.38-49.38c-12.5-12.5-12.5-32.75 0-45.25s32.75-12.5 45.25 0l103.1 103.1C414.6 241.3 416 251.1 416 256C416 260.9 414.6 270.7 406.6 278.6z\"><\/path><\/svg><\/span><span class=\"uagb-icon-list__label\">Currently, techniques utilising Reinforcement Learning are at the forefront of addressing challenges that compromise the walking capabilities of humanoid robots interacting with the friction of natural surfaces such as concrete, grass, or rubber.<\/span><\/div>\n\n\n\n<div class=\"wp-block-uagb-icon-list-child uagb-block-2a215d6f\"><span class=\"uagb-icon-list__source-wrap\"><svg xmlns=\"https:\/\/www.w3.org\/2000\/svg\" viewBox=\"0 0 512 512\"><path d=\"M256 0C114.6 0 0 114.6 0 256c0 141.4 114.6 256 256 256s256-114.6 256-256C512 114.6 397.4 0 256 0zM406.6 278.6l-103.1 103.1c-12.5 12.5-32.75 12.5-45.25 0s-12.5-32.75 0-45.25L306.8 288H128C110.3 288 96 273.7 96 256s14.31-32 32-32h178.8l-49.38-49.38c-12.5-12.5-12.5-32.75 0-45.25s32.75-12.5 45.25 0l103.1 103.1C414.6 241.3 416 251.1 416 256C416 260.9 414.6 270.7 406.6 278.6z\"><\/path><\/svg><\/span><span class=\"uagb-icon-list__label\">Researchers at the University of California, Berkeley, have devised a predictive control system for robotic locomotion that utilises reinforcement learning algorithms. This system was initially tested on a humanoid robot without onboard cameras in an indoor environment and subsequently in outdoor conditions.<\/span><\/div>\n\n\n\n<div class=\"wp-block-uagb-icon-list-child uagb-block-21ccb2d7\"><span class=\"uagb-icon-list__source-wrap\"><svg xmlns=\"https:\/\/www.w3.org\/2000\/svg\" viewBox=\"0 0 512 512\"><path d=\"M256 0C114.6 0 0 114.6 0 256c0 141.4 114.6 256 256 256s256-114.6 256-256C512 114.6 397.4 0 256 0zM406.6 278.6l-103.1 103.1c-12.5 12.5-32.75 12.5-45.25 0s-12.5-32.75 0-45.25L306.8 288H128C110.3 288 96 273.7 96 256s14.31-32 32-32h178.8l-49.38-49.38c-12.5-12.5-12.5-32.75 0-45.25s32.75-12.5 45.25 0l103.1 103.1C414.6 241.3 416 251.1 416 256C416 260.9 414.6 270.7 406.6 278.6z\"><\/path><\/svg><\/span><span class=\"uagb-icon-list__label\">Looking to the future, humanoid robots proficient in navigating and moving in challenging outdoor environments could collaborate with humans in external operations, such as emergency responses to fires and earthquakes, thereby safeguarding both people and settings.<\/span><\/div>\n<\/div><\/div>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<div style=\"height:15px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-humanoid-mobility-in-contact-with-natural-ground-friction-key-challenges\"><strong>Humanoid mobility in contact with natural ground friction: key challenges<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">We could enumerate <a href=\"https:\/\/tech4future.info\/en\/cognitive-humanoid-robots-workplaces\/\" target=\"_blank\" rel=\"noreferrer noopener\">humanoid robots<\/a> that have gained fame over the years, endowed with capabilities ranging from cognitive and interactive skills to language, visual, auditory, and tactile perceptions, and even autonomous obstacle-avoidance walking. Starting with Japan&#8217;s&nbsp;<strong>ASIMO (Advanced Step in Innovative Mobility)<\/strong>, introduced in 2000, followed by Italy&#8217;s&nbsp;<strong>iCub<\/strong>&nbsp;&#8211; launched in 2004 by the Italian Institute of Technology &#8211; and France&#8217;s&nbsp;<strong>Pepper<\/strong>&nbsp;from 2014, these are just a few of the most notable examples.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A common feature among these robots, and others like them, is their indoor operation: in laboratories, homes, factories, warehouses, restaurants, or stores. This is because&nbsp;\u00ab<strong><em>facilitating humanoid mobility on surfaces other than indoor floors, along with full-body control for trajectory tracking, poses a significant challenge.<\/em><\/strong><em>&nbsp;The difficulty stems from the need to leverage contact forces to manage leg locomotion, whilst adhering to the friction constraints of natural terrain<\/em>\u00bb&nbsp;[source: \u201c<a href=\"https:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC8414409\/\" target=\"_blank\" rel=\"noreferrer noopener\">Versatile Locomotion Planning and Control for Humanoid Robots<\/a>\u201d &#8211; National Library of Medicine, 2021].<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">More recently, a study by the Massachusetts Institute of Technology (MIT) &#8211; \u201c<a href=\"https:\/\/arxiv.org\/pdf\/2104.09025\" target=\"_blank\" rel=\"noreferrer noopener\">The MIT Humanoid Robot: Design, Motion Planning, and Control for Acrobatic Behaviors<\/a>\u201d (2021) &#8211; presents an&nbsp;<strong>acrobatic humanoid robot model<\/strong>capable of executing flips and somersaults thanks to innovative hardware design and motion planning involving two&nbsp;<strong>new proprioceptive actuators<\/strong>&nbsp;developed by MIT.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">These actuators allow the machine to orient itself in space without visual aids. However, this only reached the stage of a dynamic and realistic&nbsp;<strong>simulation<\/strong>, far removed from practical applications for real humanoid robots in natural settings.<\/p>\n\n\n\n<div style=\"height:15px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"470\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" alt=\"Image about the simulation of a backward somersault performed by the humanoid robot model developed by MIT in 2021 (Credit: \u201cThe MIT Humanoid Robot: Design, Motion Planning, and Control For Acrobatic Behaviors\u201d - https:\/\/arxiv.org\/pdf\/2104.09025).\" class=\"wp-image-16357 lazyload\" style=\"width:604px;height:auto\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" data-src=\"https:\/\/tech4future.info\/wp-content\/uploads\/2024\/05\/locomozione-robot-umanoidi-modello-mit-1024x470.png\" data-srcset=\"https:\/\/tech4future.info\/wp-content\/uploads\/2024\/05\/locomozione-robot-umanoidi-modello-mit-1024x470.png 1024w, https:\/\/tech4future.info\/wp-content\/uploads\/2024\/05\/locomozione-robot-umanoidi-modello-mit-300x138.png 300w, https:\/\/tech4future.info\/wp-content\/uploads\/2024\/05\/locomozione-robot-umanoidi-modello-mit-768x353.png 768w, https:\/\/tech4future.info\/wp-content\/uploads\/2024\/05\/locomozione-robot-umanoidi-modello-mit.png 1182w\"><figcaption class=\"wp-element-caption\">Simulation of a backward somersault performed by the humanoid robot model developed by MIT in 2021 (source: \u201cThe MIT Humanoid Robot: Design, Motion Planning, and Control For Acrobatic Behaviors\u201d &#8211; https:\/\/arxiv.org\/pdf\/2104.09025).<\/figcaption><\/figure>\n<\/div>\n\n\n<div style=\"height:15px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-humanoid-robot-locomotion-learning-based-methods\"><strong>Humanoid robot locomotion: learning-based methods<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">In recent years, despite progress in the stability of outdoor humanoid locomotion through hardware optimization and motion planning, it has been&nbsp;<strong>learning-based studies<\/strong>&nbsp;that have revitalized this area of research, focusing on the&nbsp;<strong>operational context<\/strong>&nbsp;of the robots.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Notably, in 2021, the Collaborative Robotics and Intelligent Systems Institute at Oregon State University conducted a study titled \u201c<a href=\"https:\/\/arxiv.org\/pdf\/2105.08328\" target=\"_blank\" rel=\"noreferrer noopener\">Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement Learning<\/a>\u201d. This research addressed a pivotal issue in robotic locomotion in natural environments, specifically the\u00a0<strong>accurate estimation of space needed for navigation<\/strong>, by examining the behaviour of a bipedal robot ascending and descending stairs\u00a0<strong>without any visual perception<\/strong>, as though it were blind. The technique employed was\u00a0<strong>reinforcement learning<\/strong>\u00a0(RL), a\u00a0<a href=\"https:\/\/tech4future.info\/en\/ai-decision-making-processes-reinforcement-learning\/\" target=\"_blank\" rel=\"noreferrer noopener\">machine learning strategy\u00a0that enables a machine to engage in reasoned decision-making to select precise actions needed to achieve specific objectives through interaction with its environment<\/a>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This approach, as the team elucidated, facilitated the development of a control mechanism&nbsp;<strong>through the learning of proprioceptive reflexes<\/strong>&nbsp;(enabling it to discern its spatial position without sight) and&nbsp;\u00ab<em>strategies for navigating rough terrains following exposure to various disturbances during the RL model training<\/em>\u00bb.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Nonetheless, this research was restricted to small-sized bipedal robots (distinct from humanoid-like robots) and did not extend to the practical implementation of locomotion on natural terrains. The reinforcement learning method was more tested than applied in the real world, yet it set a new course for future exploration.<\/p>\n\n\n\n<div style=\"height:15px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-reinforcement-learning-propels-outdoor-nbsp-humanoid-robot-locomotion\"><strong>Reinforcement learning propels outdoor&nbsp;humanoid robot locomotion<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Researchers at the University of California, Berkeley, have recently moved beyond conventional reinforcement learning applications with basic bipedal robots, which are limited to autonomous walking within experimental confines.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Their paper, &#8220;<a href=\"https:\/\/www.science.org\/doi\/10.1126\/scirobotics.adi9579\" target=\"_blank\" rel=\"noreferrer noopener\">Real-world Humanoid Locomotion with Reinforcement Learning<\/a>&#8220;, published in Science Robotics on 17 April 2024, debuts a&nbsp;<strong>reinforcement learning model designed to govern humanoid locomotion outside the controlled laboratory settings<\/strong>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">They have coined their developed control system a &#8220;<strong>causal transformer<\/strong>&#8220;, a term that denotes a straightforward yet effective&nbsp;<strong>neural network architecture<\/strong>. This architecture distinctively&nbsp;\u00ab<em>employs a comprehensive dataset of environmental observations and&nbsp;<\/em><strong><em>proprioceptive actions<\/em><\/strong><em>&nbsp;acquired during training to predict future actions in real-world settings, thus justifying the adjective &#8220;<\/em><strong><em>causal<\/em><\/strong><em>&#8220;<\/em>\u00bb.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The authors indicate that the&nbsp;<a href=\"https:\/\/tech4future.info\/en\/large-language-models-llm-transformer\/\" target=\"_blank\" rel=\"noreferrer noopener\">transformer<\/a>&nbsp;was trained using a vast array of video data from numerous simulations of outdoor locomotion. A notable aspect of this study is its application of the reinforcement learning model on a&nbsp;<strong>full-sized humanoid robot<\/strong>&nbsp;&#8211; standing approximately 160 cm tall and weighing 45 kg -operating&nbsp;<strong>without an onboard vision system<\/strong>, in both indoor and outdoor settings, engaging directly with environmental elements.<\/p>\n\n\n\n<div style=\"height:15px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-indoor-experiments\"><strong>Indoor experiments<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The aim of the lab tests on the humanoid robot was to specifically gauge its&nbsp;<strong>stability<\/strong>&nbsp;and robustness&nbsp;against various external forces, differing surfaces, and&nbsp;assorted loads with varying mass and shapes. Here&#8217;s an in-depth examination.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"713\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" alt=\"Image about laboratory tests on the stability of the humanoid robot controlled by the Reinforcement Learning model developed by the University of California, addressing: A) external forces B) various surfaces for walking and (C) carrying loads of different mass and shape (credit: \u201cReal-world humanoid locomotion with reinforcement learning\u201d - https:\/\/www.science.org\/doi\/10.1126\/scirobotics.adi9579).\" class=\"wp-image-16359 lazyload\" style=\"width:493px;height:auto\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" data-src=\"https:\/\/tech4future.info\/wp-content\/uploads\/2024\/05\/locomozione-robot-umanoidi-test-interni-university-california-1024x713.png\" data-srcset=\"https:\/\/tech4future.info\/wp-content\/uploads\/2024\/05\/locomozione-robot-umanoidi-test-interni-university-california-1024x713.png 1024w, https:\/\/tech4future.info\/wp-content\/uploads\/2024\/05\/locomozione-robot-umanoidi-test-interni-university-california-300x209.png 300w, https:\/\/tech4future.info\/wp-content\/uploads\/2024\/05\/locomozione-robot-umanoidi-test-interni-university-california-768x535.png 768w, https:\/\/tech4future.info\/wp-content\/uploads\/2024\/05\/locomozione-robot-umanoidi-test-interni-university-california.png 1411w\"><figcaption class=\"wp-element-caption\">Laboratory tests on the stability of the humanoid robot controlled by the Reinforcement Learning model developed by the University of California, addressing: A) external forces B) various surfaces for walking and (C) carrying loads of different mass and shape (credit: \u201cReal-world humanoid locomotion with reinforcement learning\u201d &#8211; https:\/\/www.science.org\/doi\/10.1126\/scirobotics.adi9579).<\/figcaption><\/figure>\n<\/div>\n\n\n<div style=\"height:15px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-robustness-to-external-forces\"><strong>Robustness to external forces<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Researchers at the University of California tested whether their &#8220;blind&#8221; robot, managed by the reinforcement learning model, could withstand&nbsp;<strong>sudden external forces<\/strong>&nbsp;while ambulating.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This included impacts from a large ball, abrupt pushes with a wooden stick, and resisting motions that could hinder its path. The tests confirmed that the model&#8217;s control over the humanoid could&nbsp;<strong>maintain its stability<\/strong>across each artificially constructed scenario, enabling it to&nbsp;\u00ab<strong><em>react swiftly and adjust its actions to avert falls<\/em><\/strong>\u00bb.<\/p>\n\n\n\n<div style=\"height:15px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-robustness-to-terrain-variability\"><strong>Robustness to terrain variability<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">This capability was crucial, particularly on uneven terrain, and was tested by overlaying the laboratory floor with rubber, fabric, cables, and packing materials to simulate&nbsp;<strong>different roughness levels<\/strong>, which posed tripping, slipping, or falling hazards to the robot, which had to maintain a constant forward speed of 0.15 m\/s.&nbsp;\u00ab<em>Despite these challenges, our controller enabled the robot to navigate all types of surfaces successfully. Moreover, we&nbsp;<\/em><strong><em>assessed its performance on two different slopes<\/em><\/strong><em>, with simulations featuring inclines of up to 8.7%. The results demonstrated that the robot could manage both, showing greater robustness at higher speeds (0.2 m\/s) on steeper inclines<\/em>\u00bb,&nbsp;the researchers noted.<\/p>\n\n\n\n<div style=\"height:15px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-robustness-with-various-loads\"><strong>Robustness with various loads<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The robot&#8217;s ability to maintain stability while&nbsp;<strong>carrying different types of weights<\/strong>&nbsp;as it moved forward was another tested capability by the American team. The robot was tasked with transporting an empty backpack, a full backpack, a fabric bag, a loaded garbage bag, and a paper bag, all without any balance issues.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Particularly remarkable in this last test, as highlighted by the researchers, was the reinforcement learning-based controller&#8217;s&nbsp;<strong>ability to adapt to a full garbage bag attached to its arm<\/strong>, keeping it steady to prevent it from dropping,&nbsp;\u00ab<em>despite the training involving oscillatory arm movements for body balance<\/em>\u00bb.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This is a prime example of&nbsp;<strong>behavioural adaptation to context<\/strong>&nbsp;without visual perception, encapsulating the core concept of Reinforcement Learning.<\/p>\n\n\n\n<div style=\"height:15px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-field-implementation\"><strong>Field implementation<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The outdoor trials involved the robot navigating a&nbsp;<strong>variety of settings<\/strong>&nbsp;such as squares, pedestrian crossings, sidewalks, running tracks, and grassy fields, traversing surfaces composed of diverse materials including concrete, tiles, rubber, grass, and wood, amid morning dampness and under the afternoon sun.<\/p>\n\n\n\n<div style=\"height:15px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"918\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" alt=\"Image about outdoor implementation on various types of terrain by the humanoid robot controlled by the LR model from the University of California (credit: \u201cReal-world humanoid locomotion with reinforcement learning\u201d - https:\/\/www.science.org\/doi\/10.1126\/scirobotics.adi9579).\" class=\"wp-image-16361 lazyload\" style=\"width:510px;height:auto\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" data-src=\"https:\/\/tech4future.info\/wp-content\/uploads\/2024\/05\/locomozione-robot-umanoidi-test-esterni-university-california-1024x918.png\" data-srcset=\"https:\/\/tech4future.info\/wp-content\/uploads\/2024\/05\/locomozione-robot-umanoidi-test-esterni-university-california-1024x918.png 1024w, https:\/\/tech4future.info\/wp-content\/uploads\/2024\/05\/locomozione-robot-umanoidi-test-esterni-university-california-300x269.png 300w, https:\/\/tech4future.info\/wp-content\/uploads\/2024\/05\/locomozione-robot-umanoidi-test-esterni-university-california-768x688.png 768w, https:\/\/tech4future.info\/wp-content\/uploads\/2024\/05\/locomozione-robot-umanoidi-test-esterni-university-california.png 1438w\"><figcaption class=\"wp-element-caption\">Outdoor implementation on various types of terrain by the humanoid robot controlled by the LR model from the University of California (credit: \u201cReal-world humanoid locomotion with reinforcement learning\u201d &#8211; https:\/\/www.science.org\/doi\/10.1126\/scirobotics.adi9579).<\/figcaption><\/figure>\n<\/div>\n\n\n<div style=\"height:15px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<div style=\"height:auto;width:595px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p class=\"wp-block-paragraph\">The team observed that the control system in question was trained exclusively with&nbsp;<strong>video data from simulations of movement in external environments<\/strong>. Consequently, the humanoid robot&#8217;s first real-world interactions were unprecedented. As the researchers highlighted,&nbsp;\u00ab<em>the terrain properties encountered outdoors were absent from the model&#8217;s training data. Nonetheless, the controller successfully maintained stability across all the terrains tested<\/em>\u00bb.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">During a week of comprehensive day-long outdoor tests, the robot did not suffer any falls. However, as previously mentioned, the controller is a&nbsp;<strong>causal transformer<\/strong>, relying entirely on&nbsp;<strong>historical data of environmental observations<\/strong>&nbsp;and proprioceptive responses gathered during training, and predicting subsequent actions&nbsp;<strong>based solely on this record<\/strong>, without any visual input from cameras. This occasionally led to collisions with objects or entrapment in obstacles such as steps. Nevertheless,&nbsp;\u00ab<em>it always effectively adjusted its movements to the spatial features to prevent falls<\/em>\u00bb.<\/p>\n\n\n\n<div style=\"height:15px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-glimpses-of-futures\"><strong>Glimpses of Futures<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">In the future, a predictive control system such as the one described &#8211; rooted in reinforcement learning algorithms &#8211; capable of enabling fluid, human-like ambulation outdoors for a humanoid robot even without &#8220;eyes,&#8221; could refine the behaviours and movements of humanoid robots in any terrain, no matter how challenging. This technology promises significant potential for human interaction.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">To anticipate&nbsp;<a href=\"https:\/\/tech4future.info\/en\/futures4business-en\/\" target=\"_blank\" rel=\"noreferrer noopener\">possible future scenarios<\/a>, let us employ the&nbsp;<strong>STEPS<\/strong>&nbsp;<strong>matrix<\/strong>&nbsp;to explore the impacts that the evolution of the causal transformer for controlling humanoid locomotion beyond laboratory environments could have across various domains.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>S &#8211; SOCIAL<\/strong>: in a future vista, humanoid machines increasingly sophisticated in cognitive, interactive, and perceptual capabilities, and ever more proficient in manoeuvring arms, hands, and legs, in walking and running, in ascending and descending stairs, even in the most remote and harsh open areas, could collaborate with humans &#8211; such as with Civil Protection or Law Enforcement &#8211; or even replace them in perilous operations. These operations could include outdoor activities where intervention in hazardous situations is required to protect people, animals, and environments during extreme events like fires and earthquakes. A human-sized robot, capable of using hands, arms, and legs in unison, just as a human would, while also withstanding external forces and carrying significant weights, could offer strategic support. Examples of &#8220;police robots&#8221; and &#8220;firefighter robots&#8221; &#8211; both quadrupedal and bipedal &#8211; already exist in the United States, Singapore, our own country, and globally, aiding Police Forces and Fire Brigades in external operations, though these are not humanoid (thus lacking human-like legs) and are motor-driven.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>T &#8211; TECHNOLOGICAL:<\/strong>&nbsp;the neural network devised by the University of California team, central to the humanoid control system presented, features a straightforward architecture, devoid of the recurrences and convolutions seen in deeper, more complex neural networks which demand extensive data for training. This transformer model is&nbsp;\u00ab<em>more scalable with additional data and computation, and also accommodates new input modalities<\/em>\u00bb.&nbsp;However, the future may demand a deeper neural network and other artificial intelligence techniques such as deep learning, to further refine the precision of trajectories, symmetry in leg movements, and the accuracy of their speed tracking for outdoor humanoid locomotion.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>E &#8211; ECONOMIC<\/strong>: looking forward, as technologies evolve to control the locomotion of humanoid robots even in natural settings\u2014enabling them to support professions conducted outdoors &#8211; nations will need to address the contentious issue of machine &#8211; replaced human labour. The&nbsp;<a href=\"https:\/\/www.forrester.com\/predictions\/predictions-2024\/\" target=\"_blank\" rel=\"noreferrer noopener\">Forrester Job Forecast 2020-2040<\/a>&nbsp;paints a bleak picture, predicting the loss of 12 million jobs by 2040 in sectors like retail, catering, and hospitality in countries including the United Kingdom, Germany, France, Italy, and Spain. Beyond Europe, the&nbsp;<a href=\"https:\/\/www.nysenate.gov\/legislation\/bills\/2023\/A8179\/amendment\/B\" target=\"_blank\" rel=\"noreferrer noopener\">New York State Assembly, in 2023, proposed a &#8220;robot tax&#8221;<\/a>&nbsp;on companies substituting employees with machines. As of April 2024, this bill remains mired in committee amidst vigorous debate and entrepreneurial dissent. The concept, initially introduced by&nbsp;<a href=\"https:\/\/www.forbes.com\/sites\/ianmorris\/2017\/02\/17\/tax-the-robots-says-bill-gates\/\" target=\"_blank\" rel=\"noreferrer noopener\">Bill Gates<\/a>&nbsp;in 2017 as a provocative suggestion, has been discussed in the EU, including Italy, but debates about the tax&#8217;s structure and implementation remain largely academic.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>P &#8211; POLITICAL:<\/strong>&nbsp;the advent of predictive control systems based on reinforcement learning algorithms, enabling humanoid mobility in external settings, has refocused attention on the safety of autonomous machines interacting with humans in workplaces. Notable incidents include the&nbsp;<a href=\"https:\/\/economictimes.indiatimes.com\/news\/international\/world-news\/tesla-engineer-attacked-by-robot-in-us-machine-pins-him-down-digging-metal-claws-into-workers-back-and-arm\/articleshow\/106323330.cms?from=mdr\" target=\"_blank\" rel=\"noreferrer noopener\">2021 assault on a Tesla engineer in Texas<\/a> by a humanoid robot and a fatal accident in a&nbsp;<a href=\"https:\/\/www.bbc.com\/news\/world-asia-67354709\" target=\"_blank\" rel=\"noreferrer noopener\">South Korean vegetable packing plant in November 2023<\/a>, where a robot caused a worker&#8217;s death. Regulations like those replacing the Machinery Directive 2006\/42\/EC with&nbsp;<a href=\"https:\/\/eur-lex.europa.eu\/legal-content\/IT\/TXT\/PDF\/?uri=CELEX:32023R1230\" target=\"_blank\" rel=\"noreferrer noopener\">Regulation 2023\/1230<\/a>, enforceable from January 20, 2027, impose new safety standards and protection requirements for human-machine collaboration. It is imperative that companies not only adhere to these regulations but also manage their robotic workforce responsibly to mitigate legal liabilities potentially arising from mismanagement leading to disastrous events.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>S &#8211; SUSTAINABILITY:<\/strong>&nbsp;as reinforcement learning-based control systems evolve, humanoid robots are set to expand their presence outside labs, homes, factories, warehouses, restaurants, and shops. It is critical to consider the environmental impact of artificial intelligence, the core &#8220;brain&#8221; of these machines. The impact is twofold, as highlighted in &#8220;<a href=\"https:\/\/www.sciencedirect.com\/science\/article\/abs\/pii\/S1574954123001942\" target=\"_blank\" rel=\"noreferrer noopener\">Artificial Intelligence for Carbon Emissions Using System of Systems Theory<\/a>&#8221; (Ecological Informatics, September 2023), which illustrates the paradox of AI as both a potent tool against climate change and a contributor to carbon emissions:&nbsp;\u00ab<em>one cannot deny the fact that artificial intelligence represents an effective tool in combating climate change. But neither can its role in contributing to carbon emissions be ignored<\/em>\u00bb. The emissions vary with the AI techniques employed, making it crucial to focus on sustainable AI practices that conserve power and reduce energy consumption throughout their operational life:&nbsp;\u00ab<em>although deep learning may be necessary for some specific goals, simpler machine learning algorithms often lead to similar results, but with less power and computational energy<\/em>\u00bb.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Will humanoid robots move beyond the confines of laboratories, factories, and warehouses to conquer the external world? A recent work by the University of California appears to have discovered, through an innovative reinforcement learning model, the key to this transition.<\/p>\n","protected":false},"author":9,"featured_media":16356,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_uag_custom_page_level_css":"","_lmt_disableupdate":"","_lmt_disable":"","footnotes":""},"categories":[1099],"tags":[],"class_list":["post-16384","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-tech-en"],"acf":[],"modified_by":"Paola Cozzi","uagb_featured_image_src":{"full":["https:\/\/tech4future.info\/wp-content\/uploads\/2024\/05\/locomozione-robot-umanoidi-terreni-naturali.jpg",827,368,false],"thumbnail":["https:\/\/tech4future.info\/wp-content\/uploads\/2024\/05\/locomozione-robot-umanoidi-terreni-naturali-150x150.jpg",150,150,true],"medium":["https:\/\/tech4future.info\/wp-content\/uploads\/2024\/05\/locomozione-robot-umanoidi-terreni-naturali-300x133.jpg",300,133,true],"medium_large":["https:\/\/tech4future.info\/wp-content\/uploads\/2024\/05\/locomozione-robot-umanoidi-terreni-naturali-768x342.jpg",768,342,true],"large":["https:\/\/tech4future.info\/wp-content\/uploads\/2024\/05\/locomozione-robot-umanoidi-terreni-naturali.jpg",827,368,false],"1536x1536":["https:\/\/tech4future.info\/wp-content\/uploads\/2024\/05\/locomozione-robot-umanoidi-terreni-naturali.jpg",827,368,false],"2048x2048":["https:\/\/tech4future.info\/wp-content\/uploads\/2024\/05\/locomozione-robot-umanoidi-terreni-naturali.jpg",827,368,false],"inner-slider":["https:\/\/tech4future.info\/wp-content\/uploads\/2024\/05\/locomozione-robot-umanoidi-terreni-naturali.jpg",827,368,false],"featured-blog":["https:\/\/tech4future.info\/wp-content\/uploads\/2024\/05\/locomozione-robot-umanoidi-terreni-naturali-543x357.jpg",543,357,true],"large-blog":["https:\/\/tech4future.info\/wp-content\/uploads\/2024\/05\/locomozione-robot-umanoidi-terreni-naturali-524x276.jpg",524,276,true],"medium-blog":["https:\/\/tech4future.info\/wp-content\/uploads\/2024\/05\/locomozione-robot-umanoidi-terreni-naturali-415x276.jpg",415,276,true],"related-large":["https:\/\/tech4future.info\/wp-content\/uploads\/2024\/05\/locomozione-robot-umanoidi-terreni-naturali-343x248.jpg",343,248,true],"related-small":["https:\/\/tech4future.info\/wp-content\/uploads\/2024\/05\/locomozione-robot-umanoidi-terreni-naturali-343x198.jpg",343,198,true],"normal-blog":["https:\/\/tech4future.info\/wp-content\/uploads\/2024\/05\/locomozione-robot-umanoidi-terreni-naturali-306x198.jpg",306,198,true],"small-blog":["https:\/\/tech4future.info\/wp-content\/uploads\/2024\/05\/locomozione-robot-umanoidi-terreni-naturali-95x96.jpg",95,96,true],"small-author":["https:\/\/tech4future.info\/wp-content\/uploads\/2024\/05\/locomozione-robot-umanoidi-terreni-naturali-70x70.jpg",70,70,true]},"uagb_author_info":{"display_name":"Paola Cozzi","author_link":"https:\/\/tech4future.info\/en\/author\/paolacozzi\/"},"uagb_comment_info":0,"uagb_excerpt":"Will humanoid robots move beyond the confines of laboratories, factories, and warehouses to conquer the external world? A recent work by the University of California appears to have discovered, through an innovative reinforcement learning model, the key to this transition.","meta_box":{"standard":[{"title":"Humanoid mobility in contact with natural ground friction: key challenges","link":"#h-humanoid-mobility-in-contact-with-natural-ground-friction-key-challenges"},{"title":"Humanoid robot locomotion: learning-based methods","link":"#h-humanoid-robot-locomotion-learning-based-methods"},{"title":"Reinforcement learning propels outdoor humanoid robot locomotion","link":"#h-reinforcement-learning-propels-outdoor-humanoid-robot-locomotion"},{"title":"Indoor experiments","link":"#h-indoor-experiments"},{"title":"Field implementation","link":"#h-field-implementation"},{"title":"Glimpses of Futures","link":"#h-glimpses-of-futures"}],"content":false,"name":false,"role":false,"type":false,"category":false,"count":false,"btn":false,"link":false,"layout":false,"title":false,"director":false,"user":false,"email":false,"posts":false,"description":false,"form":false,"subtitle":false,"innertitle":false,"background":false},"_links":{"self":[{"href":"https:\/\/tech4future.info\/en\/wp-json\/wp\/v2\/posts\/16384","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/tech4future.info\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/tech4future.info\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/tech4future.info\/en\/wp-json\/wp\/v2\/users\/9"}],"replies":[{"embeddable":true,"href":"https:\/\/tech4future.info\/en\/wp-json\/wp\/v2\/comments?post=16384"}],"version-history":[{"count":0,"href":"https:\/\/tech4future.info\/en\/wp-json\/wp\/v2\/posts\/16384\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/tech4future.info\/en\/wp-json\/wp\/v2\/media\/16356"}],"wp:attachment":[{"href":"https:\/\/tech4future.info\/en\/wp-json\/wp\/v2\/media?parent=16384"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/tech4future.info\/en\/wp-json\/wp\/v2\/categories?post=16384"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/tech4future.info\/en\/wp-json\/wp\/v2\/tags?post=16384"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}