A Reinforcement Learning Approach to Automatic Voltage Regulator System

Abstract

An Automatic Voltage Regulator (AVR) system utilized to keep the terminal voltage of a synchronous generator at the desired level has received much attention among researchers. Designing an efficient and robust control scheme for the AVR system to maintain a specified voltage level is an important research area. From the control area perspective, reinforcement learning, an adaptive optimal control method, has received increasing attention in reference tracking problems. This article discusses a reinforcement learning approach to an AVR system and its experimental validation. A deep deterministic policy gradient (DDPG) agent working in continuous-time is designed offline to improve dynamic system characteristics of the AVR system besides its robustness against load disturbance, parameter uncertainties, and reference change. In the DDPG agent design process, the limits of the produced control signal are taken into account to perform a feasible simulation similar to a real-time application. The performance of the proposed learning-based controller is analyzed in three categories: transient and steady-state responses, stability analysis, and robustness analysis against parameter uncertainties, reference change, and load disturbance. A comparison with recently published papers employing Fuzzy-PID, PID-F, (PIDNDN2)-D-lambda-N-2, PIDD2 ,and PID controllers in which various heuristic optimization algorithms were employed to optimally tune the controller parameters is made. Furthermore, to demonstrate that the behavior of the learning-based approach provides a stable and satisfactory performance, it is analyzed for a real synchronous generator connected to a 230 kV network using Matlab/Simulink environment. The results presented in this paper indicate that the proposed learning-based controller ensures the stability of the AVR system, significantly improves the regulating performance, and most impressively, is robust against parameter uncertainties, reference change, and load disturbance.