Model Predictive Control-Based Reinforcement Learning

Reinforcement Learning (RL) has garnered much attention in the field of control due to its capacity to learn from interactions and adapt to complex and dynamic environments. However, RL is challenging because it needs to balance exploration, seeking new strategies, and exploitation, leveraging known strategies for maximum gain. To address these challenges, this paper proposes a Model Predictive Control (MPC) based RL approach, where the state value function in RL is utilized as the cost function in MPC, and the system dynamic model is represented by neural networks (NNs). This eliminates the need for human intervention and addresses inaccuracies in the system model. Additionally, MPC-guided RL accelerates convergence during RL training, thereby enhancing sample efficiency. Reported results demonstrate that the proposed method outperforms traditional RL algorithms and does not require prior knowledge of the system.