Off-road hybrid electric vehicle energy management strategy using multi-agent soft actor-critic with collaborative-independent algorithm

Hybrid electric vehicles (HEVs) reduce carbon emissions and save energy, and hybrid energy storage system (HESS) consist of a battery and a supercapacitor which has high energy density and high power density. The HEV equipped with HESS performs better in off-road conditions than single energy storage source. However, its energy management requires multiple input and multiple output (MIMO) control. In this paper, a multi-agent soft actor-critic (MASAC) based energy management strategy (EMS) is proposed to solve the multi-objective optimizing problem considering fuel economy, maintaining state of charge (SOC) and reducing battery state of health (SOH) decay. MASAC based EMS has two advantages: 1) it decomposed the search space into two subspaces, improving the learning efficiency. 2) a novel collaborative-independent algorithm is proposed to allocate rewards among agents, thereby improving the learning stability. Thus, the optimal actions are efficiently and collaboratively learned by two agents, engine agent and HESS agent, showing better performance in multi-objective optimization. In the simulation, the proposed EMS is compared with dynamic programming (DP) and soft actor-critic (SAC) in both off-road driving cycle and standard driving cycles. Simulation results show that the proposed collaborative-independent algorithm enhances the learning efficiency and learning stability of MASAC, while improving the real-time performance of EMS. In off-road conditions, the equivalent fuel consumption of MASAC is slightly better than that of DP. The SOH decay of MASAC is only 20 % higher than DP, significantly outperforming SAC. Furthermore, MASAC demonstrates superior performance in three standard working cycles when compared with SAC.