------------------------------------------------------
Ernesto Martinez, Nottingham University:

Batch Process Automation using a Re-inforcement 
Learning/Fuzzy Logic Based Approach
------------------------------------------------------

                 
	Abstract

The degree of automation of batch process units is still very low in many 
industries. As a result, the key to success in achieving products of high 
and reproducible quality depends to a great extent on experienced human 
operators. In the current economic climate characterised by intense 
competition and a shortening of product cycle life, such a dependence is 
unsatisfactory and worrisome.

The talk will adress the issues associated with this state of affairs and 
points out some qualities of a human being that could help finding a new 
avenue to the problem. Mainly, the capability of learning by interaction 
with a system or a simulator of it seems to be the key to achieve a 
breakthrough in batch process automation. Learning by interaction has been 
formalised recently in a framework known as Re-inforcement Learning. So the 
application of this new framework to increase the autonomous operation of a 
batch unit  will be discussed. To exemplify the ideas, I will use a semi-batch 
reactor where an autocatalytic reaction takes place and a process where 
reaction and distillation have been integrated.

Re-inforcement Learning (RL) is a broad class of optimal control methods 
aimed to learn sequences of actions (policies) from experience, simulation 
or direct interaction with the system to be controlled. Designing a RL 
controller revolves around the problem faced by an agent that must learn 
to carry out tasks through trial-and-error interactions with the system 
in which is embedded; reward/punishment is provided for each control action 
taken by the controller (agent) at each process state. The core elements 
for quick learning using a global control strategy are the search mechanism 
and the memory mechanism of the RL learning controller. In the talk the use 
of fuzzy logic for implementing both search and memory schemes will be 
proposed. Results obtained with the new RL algorithm will be presented.