Dynamic and adaptive learning for autonomous decision-making in beyond visual range air combat

The environment of beyond-visual-range (BVR) air combat is complex and dynamic, making traditional decision-making methods insufficient for modern combat scenarios. This paper first analyzes the confrontation process in BVR air combat and develops a corresponding decision-making model for air combat. To address the challenge of coupling maneuver and missile launch decisions, we propose a hybrid bifurcation action space design method, allowing for more precise control and improved learning. Additionally, this paper introduces Progressive Opponent Reinforcement Learning (PORL), which incorporates progressively challenging opponents to simulate real-world adversary strategies. Based on the Soft Actor-Critic (SAC) algorithm, this method strengthens the exploration and utilization of learning balance through maximum entropy, and dynamically adjusts the opponent's tactics according to the agent's performance, thus improving the agent's learning efficiency and adaptability in the rapidly changing confrontation environment. Furthermore, a dynamic opponent sampling mechanism is designed to select adversaries with varying difficulty levels based on the agent's current performance, ensuring a balanced training process. Simulation results demonstrate that the proposed decision-making framework significantly improves the autonomous decision-making capabilities and countermeasure effectiveness of agents in BVR air combat.