Reinforcement learning in robotics

What it means

Reinforcement learning, or RL, learns behavior through interaction with an environment. An agent receives observations, selects actions and receives reward signals. The objective is to improve cumulative reward, often called return, rather than only match a supplied example action. Spinning Up defines a policy as the rule the agent uses to choose actions. [1]

In robotics, the environment can be simulated or physical. The action might be a joint target or another control command, depending on the system. A policy's input, output and training objective are therefore necessary context for interpreting the phrase RL-controlled robot. [1, 2]

How it applies to humanoids

Humanoid-Gym trains locomotion skills in Isaac Gym and includes verification in MuJoCo before physical testing. Its paper reports transfer to RobotEra's XBot-S and XBot-L platforms. The result concerns the evaluated locomotion setup, not a general guarantee of household manipulation or industrial reliability. [2]

Unitree's official repository supports reinforcement-learning workflows for G1, H1 and other robots. It lists training to maximize designed rewards, policy playback, sim-to-sim testing and physical deployment as separate stages. G1 is linked here because its record includes that official learning repository. [3]

Terminology and evaluation traps

Reward is a chosen training objective. A large return only says that a policy did well under that reward definition; it is not automatically a measure of usefulness in a different task. Compare reward formulas and evaluation conditions before comparing scores from different experiments. [1]

Training and execution are different phases. The presence of a learned policy on a robot does not establish that the robot continues updating it during operation. Unitree distinguishes training from playback and deployment, making this distinction visible in a practical pipeline. [3]

Zero-shot sim-to-real describes transfer without the specified additional real-world training step. It does not mean the system had no training, no engineering or no simulator-specific preparation. Humanoid-Gym's reported physical experiments follow simulation training and cross-simulator verification. [2]

How to read it on humanoidversus.com

The ai_stack field records disclosed model or controller names; a null does not show that RL is absent. compute records hardware and is not a training-quality score. Read official repositories and technical papers for what was actually trained.

This entry has no live table. On benchmark pages, compare results only for the same task, metric and protocol. Robot speed or torque should retain their own product-source conditions rather than inherit a controller paper's return. A research policy running on a robot also does not change its commercial availability.

Further reading

Spinning Up supplies the agent, policy and return terminology. Humanoid-Gym explains a humanoid locomotion framework and evaluation scope. Unitree's repository shows how training, simulator checks and hardware deployment are separated in practice. [1–3]