TY - BOOK

T1 - Numerical methods for the solution of optimal feedback control problems

AU - Richardson, Steven

PY - 2007

Y1 - 2007

N2 - In this thesis we consider some numerical solution methods for solving optimal feed-back control problems. Finding solutions to problems of this nature involves a significantly increased degree of difficulty compared to the related task of solving optimal open-loop control problems. Specifically, a feed-back control depends on both time and state variables, and so its determination by numerical schemes is subject to the well known "curse of dimensionality". Consequently efficient numerical methods are critical to the accurate determination of optimal feed-back controls. Optimal feed-back control problems can be formulated in two equivalent ways, providing the opportunity to seek solutions using either of these alternatives. The first formulation expresses the problem as a nonlinear hyperbolic partial differential equation, known as the Hamilton-Jacobi-Bellman (HJB) equation. We present a method based on solving the so called viscosity approximation to the HJB equation, in which the HJB equation is perturbed by a small viscosity term to give a quasi-linear parabolic partial differential equation. The method involves the use of an exponentially fitted finite volume/element method in the spatial domain, combined with an implicit time stepping, to produce an unconditionally stable solution scheme. We also consider some of the more theoretical aspects related to the solution of the HJB equation using the exponentially fitted finite volume/element method. In particular, we present relevant existence, uniqueness and convergence results. The second formulation, often referred to as the direct approach, considers the optimisation of an objective functional with respect to the control function. The optimisation is subject to the system dynamics, and various constraints on the state and control variables. We consider an approach based on this direct formulation using a modified version of the Multivariate Adaptive Regression B-spline algorithm (BMARS), applied previously in high dimensional regression modeling. This method was developed specifically to enable the solution of optimal feed-back control problems with high dimensional state spaces. We demonstrate the efficiency of the approach by performing numerical experiments on problems with up to six state variables.

AB - In this thesis we consider some numerical solution methods for solving optimal feed-back control problems. Finding solutions to problems of this nature involves a significantly increased degree of difficulty compared to the related task of solving optimal open-loop control problems. Specifically, a feed-back control depends on both time and state variables, and so its determination by numerical schemes is subject to the well known "curse of dimensionality". Consequently efficient numerical methods are critical to the accurate determination of optimal feed-back controls. Optimal feed-back control problems can be formulated in two equivalent ways, providing the opportunity to seek solutions using either of these alternatives. The first formulation expresses the problem as a nonlinear hyperbolic partial differential equation, known as the Hamilton-Jacobi-Bellman (HJB) equation. We present a method based on solving the so called viscosity approximation to the HJB equation, in which the HJB equation is perturbed by a small viscosity term to give a quasi-linear parabolic partial differential equation. The method involves the use of an exponentially fitted finite volume/element method in the spatial domain, combined with an implicit time stepping, to produce an unconditionally stable solution scheme. We also consider some of the more theoretical aspects related to the solution of the HJB equation using the exponentially fitted finite volume/element method. In particular, we present relevant existence, uniqueness and convergence results. The second formulation, often referred to as the direct approach, considers the optimisation of an objective functional with respect to the control function. The optimisation is subject to the system dynamics, and various constraints on the state and control variables. We consider an approach based on this direct formulation using a modified version of the Multivariate Adaptive Regression B-spline algorithm (BMARS), applied previously in high dimensional regression modeling. This method was developed specifically to enable the solution of optimal feed-back control problems with high dimensional state spaces. We demonstrate the efficiency of the approach by performing numerical experiments on problems with up to six state variables.

KW - Finite volume method

KW - Control theory

KW - Numerical calculations

KW - Spline theory

KW - Optimal feedback control

KW - B-spline approximation

KW - Hamilton-Jacobi-Bellman equation

M3 - Doctoral Thesis

ER -