Deep Reinforcement Learning-Based Collision-Free Navigation for Magnetic Helical Microrobots in Dynamic Environments

Magnetic helical microrobots have great potential in biomedical applications due to their ability to access confined and enclosed environments via remote manipulation by magnetic fields. However, achieving collision-free navigation for microrobots in complex and unstructured environments, particularly in highly dynamic settings, remains a challenge. In this paper, we present a novel deep reinforcement learning-based control framework for magnetic helical microrobots, focusing on the tasks of goal-reaching and dynamic obstacle avoidance. To streamline data collection, a specialized training environment capturing essential aspects of navigation for magnetic helical microrobots is devised. The robustness and adaptability of the trained policy are supported using a randomization technique within the training environment. To facilitate seamless integration with real-world magnetic actuation systems, a visual processing algorithm based on OpenCV is devised and incorporated to collect policy observations. Simulations and experiments in various scenarios validate the high robustness and adaptability of the method. The performance assessment revealed a success rate of 99% in navigating the microrobot around 4 dynamic obstacles of comparable speeds and a success rate of 90% in environments with 14 dynamic obstacles. The results indicate the potential for future applications of our method in unstructured, confined, and dynamic living environments.