Multi-agent Recurrent Actor-Critic for Cooperative Decision-Making in Within Visual Range Air Combat

In recent years, the significance of cooperative decision-making in autonomous air combat scenarios has gained widespread recognition. Consequently, this paper introduces an innovative algorithm named Multi-Agent Recurrent Actor-Critic (MARAC), explicitly designed to enhance cooperative decision-making in autonomous within visual range (WVR) air combat. By leveraging the Centralized-Training-Distributed-Execution (CTDE) framework and utilizing recurrent neural networks, the MARAC algorithm improves the efficacy of communication-independent cooperative air combat strategies, resulting in more effective outcomes. Furthermore, the incorporation of curriculum learning (CL) and self-play (SP) techniques is proposed to boost the algorithm’s learning efficiency. Experimental results demonstrate that the MARAC algorithm significantly enhances the performance of cooperative decision-making by effectively addressing challenges associated with partial observations and complex confrontation dynamics.