Module lq_regulator 

Finite-horizon linear-quadratic regulator.

A validating instance of ControlProblem with a closed-form solution obtained from the Riccati equation. The state is x in R^n and the control is u in R^m.

Dynamics

dx = (A x + B u) dt + C dW

where C is an n x n diffusion matrix (constant, with independent noise factors mapped through C).

Objective

Minimize the expected running quadratic cost plus terminal quadratic cost:

J(u) = E[ integral_0^T ( x' Q x + u' R u ) dt + x_T' Q_T x_T ]

Equivalently, maximize -J(u), which is the sign convention used by ControlProblem. Q, Q_T, and R are positive semidefinite (positive definite for R) so the problem is well posed.

HJB equation

0 = d_t V + sup_u { -(x' Q x + u' R u)
     + (A x + B u)' grad V + 0.5 tr(C C' hess V) }

Exact solution

The value is V(t, x) = -x' P(t) x - q(t), where P(t) solves the backward Riccati ODE

dP/dt = -A' P - P A - Q + P B R^{-1} B' P,   P(T) = Q_T
dq/dt = -tr(C C' P),                          q(T) = 0

and the optimal control is the linear feedback law

u*(t, x) = -R^{-1} B' P(t) x

q(t) is independent of the state; it contributes only the noise-induced (diffusion) correction to the value and cancels out of the control.

Structs

LqRegulator

Finite-horizon linear-quadratic regulator with constant coefficients.