Sim-to-Real of Humanoid Locomotion Policies via Joint Torque Space Perturbation Injection
Summary: arXiv:2603.21853v2 Announce Type: replace-cross
Abstract
This paper proposes a novel alternative to existing sim-to-real methods for training control policies with simulated experiences. Unlike prior methods that typically rely on domain randomization over a fixed finite set of parameters, the proposed approach injects state-dependent perturbations into the input joint torque during forward simulation. These perturbations are designed to simulate a broader spectrum of reality gaps than standard parameter randomization without requiring additional training. By using neural networks as flexible perturbation generators, the proposed method can represent complex, state-dependent uncertainties, such as nonlinear actuator dynamics and contact compliance, that parametric randomization cannot capture. Experimental results demonstrate that the proposed approach enables humanoid locomotion policies to achieve superior robustness against complex, unseen reality gaps in both simulation and real-world deployment.
Introduction
Recent advancements in robotics have heightened the demand for effective simulation-to-reality (sim-to-real) transfer techniques. Traditional methods often depend on domain randomization, which adjusts a limited set of parameters to mimic real-world conditions. However, these approaches can fall short in capturing the intricacies of real-world dynamics.
Proposed Methodology
The proposed approach introduces a fresh perspective by incorporating joint torque space perturbation injection during the simulation phase. Key features of this methodology include:
- State-Dependent Perturbations: Unlike fixed parameter adjustments, the perturbations are tailored to the current state of the robot, enhancing the realism of the simulation.
- Neural Network Utilization: Neural networks serve as dynamic generators for perturbations, allowing for the representation of complex uncertainties that traditional methods cannot achieve.
- Robustness Against Reality Gaps: The method aims to improve policy robustness when faced with unforeseen challenges in real-world environments.
Experimental Results
To validate the effectiveness of the proposed method, experiments were conducted comparing traditional domain randomization techniques with the new perturbation injection method. Results indicated a marked improvement in the performance and adaptability of humanoid locomotion policies, highlighting the following outcomes:
- Increased stability during locomotion in both simulated and real environments.
- Enhanced capability to navigate complex terrains without prior exposure.
- Lower failure rates when confronted with unexpected disturbances, such as uneven surfaces or varying frictional properties.
Conclusion
The findings from this research underscore the potential of joint torque space perturbation injection as a transformative method for improving sim-to-real transfer in robotic applications. By effectively simulating a wider range of uncertainties, this approach not only enhances the performance of humanoid locomotion policies but also sets a new benchmark for future research in the field of robotics.
Researchers and practitioners in the field are encouraged to explore this methodology further, as it offers a promising avenue for bridging the gap between simulated environments and real-world applications.