Model-free Intelligent Control using Reinforcement Learning and Temporal Abstraction-applied to pH Control
Authors: | Syam Syafiie, University of Valladolid, Spain Tadeo Fernando, University of Valladolid, Spain Martinez Ernesto, Consejo Nacional de Investigaciones Científicas y Técnicas, Argentina |
---|
Topic: | 1.2 Adaptive and Learning Systems |
---|
Session: | Applications of Adaptive and Learning Control |
---|
Keywords: | learning control, intelligent control, online learning, agents, process control, neutralization process, pH control, temporal abstraction, model free |
---|
Abstract
This article presents a solution to pH control based on model-free intelligent control (MFIC) using reinforcement learning. This control technique is proposed because the algorithm gives a general solution for acid-base system, yet simple enough for its implementation in existing control hardware. In standard reinforcement learning, the interaction between an agent and the environment is based on a fixed time scale: during learning, the agent can select several primitive actions depending on the system state. A novel solution is presented, using multi-step actions (MSA): actions on multiple time scales consist of several identical primitive actions. This solves the problem of determining a suitable fixed time scale to select control actions so as to trade off accuracy in control against learning complexity. The application of multi-step actions on a simulated pH process shows that the proposed MFIC learns to control adequately the neutralization process. Copyright © 2005 IFAC