A Hierarchical Spatial-Aware Algorithm with Efficient Reinforcement Learning for Human-Robot Task Planning and Allocation in Production
Summary: arXiv:2604.12669v1 Announce Type: new
Abstract: In advanced manufacturing systems, humans and robots collaborate to conduct the production process. Effective task planning and allocation (TPA) is crucial for achieving high production efficiency, yet it remains challenging in complex and dynamic manufacturing environments. The dynamic nature of humans and robots, particularly the need to consider spatial information (e.g., humans’ real-time position and the distance they need to move to complete a task), substantially complicates TPA.
To address the above challenges, we decompose production tasks into manageable subtasks. We then implement a real-time hierarchical human-robot TPA algorithm, including a high-level agent for task planning and a low-level agent for task allocation. For the high-level agent, we propose an efficient buffer-based deep Q-learning method (EBQ), which reduces training time and enhances performance in production problems with long-term and sparse reward challenges. For the low-level agent, a path planning-based spatially aware method (SAP) is designed to allocate tasks to the appropriate human-robot resources, thereby achieving the corresponding sequential subtasks.
We conducted experiments on a complex real-time production process in a 3D simulator. The results demonstrate that our proposed EBQ&SAP method effectively addresses human-robot TPA problems in complex and dynamic production processes.
Key Features of the Proposed Method
- Hierarchical Structure: The algorithm is designed with a high-level task planning agent and a low-level task allocation agent, making it adaptable to various manufacturing scenarios.
- Efficient Buffer-Based Deep Q-Learning (EBQ): This innovative approach significantly reduces training time while improving performance in scenarios characterized by long-term and sparse rewards.
- Spatial Awareness: The method incorporates real-time spatial information, allowing for more effective task allocation based on the current positions of both humans and robots.
- Task Decomposition: By breaking down complex tasks into smaller subtasks, the algorithm simplifies the planning and allocation process, enhancing overall production efficiency.
Implications for the Manufacturing Industry
The introduction of the EBQ&SAP method has significant implications for the manufacturing industry:
- Increased Efficiency: By optimizing task planning and allocation, manufacturers can enhance productivity, reduce downtime, and ultimately improve overall operational efficiency.
- Improved Human-Robot Collaboration: The proposed method fosters better collaboration between human workers and robotic systems, ensuring that both resources are utilized effectively.
- Scalability: The algorithm’s hierarchical approach allows for scalability in various manufacturing environments, making it suitable for both small-scale and large-scale operations.
- Adaptability: The real-time nature of the algorithm ensures that it can adapt to changing conditions in the manufacturing process, accommodating fluctuations in workforce availability and task demands.
Conclusion
In conclusion, the hierarchical spatial-aware algorithm with efficient reinforcement learning presents a promising solution to the challenges of human-robot task planning and allocation in modern manufacturing systems. By leveraging advanced techniques in deep learning and spatial awareness, this method not only enhances productivity but also sets a new standard for collaboration between humans and robots in production environments.