A TD3 Algorithm Based Reinforcement Learning Controller for DC-DC Switching Converters

Jian Ye, Huanyu Guo, Sen Mei, Yingjie Hu, Xinan Zhang

Research output: Chapter in Book/Conference paperConference paperpeer-review

5 Citations (Scopus)

Abstract

Various linear and nonlinear controllers have been developed to improve the dynamic performance of DC-DC converters. Most controllers can only be designed on the basis of understanding the mathematical model of DC-DC converter, but the inherent nonlinear and time-varying characteristics of DC-DC switching converter make it difficult to complete the precise modeling, so the model-based control design is complex and the control performance is limited. In order to overcome the problem, this paper proposes a reinforcement learning (RL) controller based on the twin-delayed deep deterministic policy gradient (TD3) algorithm. This controller does not need the model of the switching converter. The converter will be regarded as a black box model, the policy approximation function (policy neural network) can be trained and learned by constructing a Markov decision process interacting with the black box model in the control system, and the optimal control action can be output. The RL controller is developed based on actor critic architecture, and a TD3 algorithm with higher learning efficiency is proposed to improve the control performance of the RL controller. The proposed RL controller based on TD3 algorithm is compared with the traditional PI controller. The simulation results show that the RL controller has better dynamic performance when the converter starts and the load step changes.

Original languageEnglish
Title of host publication2023 International Conference on Power Energy Systems and Applications, ICoPESA 2023
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages358-363
Number of pages6
ISBN (Electronic)9798350345605
DOIs
Publication statusPublished - 5 Jun 2023
Event2023 International Conference on Power Energy Systems and Applications - Nanjing, China
Duration: 24 Feb 202326 Feb 2023

Publication series

Name2023 International Conference on Power Energy Systems and Applications, ICoPESA 2023

Conference

Conference2023 International Conference on Power Energy Systems and Applications
Abbreviated titleICoPESA 2023
Country/TerritoryChina
CityNanjing
Period24/02/2326/02/23

Fingerprint

Dive into the research topics of 'A TD3 Algorithm Based Reinforcement Learning Controller for DC-DC Switching Converters'. Together they form a unique fingerprint.

Cite this