Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: THEend8_
ANALYSIS AND OPTIMIZATION 63
7. Control Theory
In control theory, we study problems of the form
max
x,u
ˆ t1
t0
f(t, x, u) dt,
where x = x(t) is called the state and u = u(t) is called the control, subject to
x˙ = g(t, x, u), u(t) œ U µ R, x(t0) = x0,
and
(a)x(t1) = x1, (b)x(t1) Ø x1, (c)x(t1) = free, or (d)x(t1) Æ x1.
The set U is called the control region.
Definition 7.1. A pair (x, u) that satisfies the conditions
x˙ = g(t, x, u), u(t) œ U µ R, x(t0) = x0,
and, depending on the problem,
(a)x(t1) = x1, (b)x(t1) Ø x1, (c)x(t1) = free, or (d)x(t1) Æ x1
is a called an admissible pair. An optimal pair is an admissible pair that
maximizes the integral ˆ t1
t0
f(t, x, u) dt,
among all admissible pairs.
As a motivating example, let’s consider an economy evolving over time, where
k = k(t) represents the capital stock,
f = f(k) represents production, and
s = s(t) represents the fraction of production set aside for investment.
Then,
(1≠ s(t))f(k(t)) represents consumption per unit time.
If we wanted to maximize consumption over a given period, we would have the
control problem
max
k,s
ˆ T
0
(1≠ s)f(k) dt,
where k = k(t) is our state and s = s(t) is our control, subject to
k˙ = sf(k), s(t) œ [0, 1], k(0) = k0, and k(T ) Ø kT .
Here, k0 represents the amount of stock we start with, and kT represents the amount
of stock we want to ensure exists at time T .
Like a Lagrangian, we introduce an auxiliary function called a Hamiltonian to
help us solve control theory problems, also known as optimal control problems.
Definition 7.2. For p0 œ R and p = p(t), we define the Hamiltonian
H(t, x, u, p) = p0f(t, x, u) + p(t)g(t, x, u).
The function p = p(t) is called the adjoint function.
Remark 7.3. We can assume that p0 is either equal to 0 or 1. Indeed, if p0 ”= 0,
then divide everything by p0, and redefine the adjoint function as p/p0.
64 ANALYSIS AND OPTIMIZATION
Theorem 7.4 (The Maximum Principle). Let (xú, uú) be an optimal pair. Then,
there is a continuous function pú and a number p0 œ {0, 1}, i.e, p0 = 0 or p0 = 1,
such that
1. (p0, pú(t)) ”= (0, 0) for all t œ [t0, t1].
2. uú maximizes H with respect to u, i.e.,
H(t, xú(t), u, pú(t)) Æ H(t, xú(t), uú(t), pú(t)) for all u œ U.
3. p˙ú = ≠ˆxH(t, xú(t), uú(t), pú(t)).
4. pú satisfies the following transversality condition at t = t1, depending on
the problem
(a)nothing, (b) pú(t1) Ø 0 and pú(t1) = 0 if xú(t1) > x1.
(c) pú(t1) = 0, (d) pú(t1) Æ 0 and pú(t1) = 0 if xú(t1) < x1.
Theorem 7.5 (Mangasarian). Let (xú, uú) be an admissible pair. Suppose that 1.
through 4. of the maximum principle are satisfied for some pú with p0 = 1. If U , the
control region, is convex and H(t, x, u, pú(t)) is concave in (x, u) for all t œ [t0, t1],
then (xú, uú) is an optimal pair.
Given these two theorems, how do we solve an optimal control problem?
Step 0: Make sure the problem is in standard form; typically this just involves
turning a minimization problem into a maximization problem.
Step 1: (a) Identify the Hamiltonian, and set p0 = 1. (b) For each triplet (t, x, p),
maximize H(t, x, u, p) with respect to u over U , and set uˆ = uˆ(t, x, p) to be the
maximizer.
Step 2: Find particular solutions to the ODEs, for x = x(t) and p = p(t),
x˙ = g(t, x, uˆ(t, x, p))
and
p˙ = ≠ˆxH(t, x, uˆ(t, x, p), p)
using the (1) boundary conditions and (2) the transversality condition to go from
the general solution to a particular solution. Call these particular solutions xú and
pú. Then, set uú = uú(t) = uˆ(t, xú(t), pú(t)).
Step 3: Check the sucient conditions from Mangasarian hold, i.e., (1) U is
convex and (2) H(t, x, u, pú(t)) is concave in (x, u) for all t œ [t0, t1]. If these two
things hold, (xú, uú) is an optimal pair.
For example, let’s consider the optimal control problem
min
x,u
ˆ 1
0
u2 dt
subject to
x˙ = u+ ax, x(0) = 1, x(1) = free, and u œ U = R.
Here a œ R is a fixed but arbitrary constant.
Step 0: In standard form, our problem is
max
x,u
ˆ 1
0
≠u2 dt
subject to
x˙ = u+ ax, x(0) = 1, x(1) = free, and u œ U = R.
ANALYSIS AND OPTIMIZATION 65
Step 1: The Hamiltonian with p0 = 1 is
H(t, x, u, p) = ≠u2 + p(u+ ax).
To maximize H(t, x, u, p) with respect to u œ U = R, we note that H(t, x, u, p) is
concave in u. So stationary points with respect to u are maxima. Therefore, we
compute
0 = ˆuH = ≠2u+ p.
And we find that
uˆ = p2 .
Step 2: Our ODEs are
x˙ = p2 + ax and p˙ = ≠pa.
So p = C1e≠at. Using the transversality condition, p(1) = 0 (since x(1) = free), we
find that C1 = 0. Thus,
pú(t) = 0 for all t.
Plugging this into the ODE for x, we get the simpler ODE
x˙ = ax,
which is solved by x = C2eat. Using the boundary condition, x(0) = 1, we see that
C2 = 1. And so,
xú(t) = eat.
In turn,
uú(t) = uˆ(t, xú(t), pú(t)) = p
ú(t)
2 = 0 for all t.
So our candidate solution is
(xú, uú) = (eat, 0).
Step 3: Since R is convex, U = R is convex. In addition, H(t, x, u, pú(t)) = ≠u2,
which is concave in (x, u) for all t œ [t0, t1]. So the hypothesis of Mangasarian are
satisfied, which implies that our candidate solution is an optimal solution, i.e.,
(xú, uú) = (eat, 0)
is an optimal pair / maximizing pair.
Now, let’s redo the same problem, but with a dierent terminal boundary
condition. Let’s consider the optimal control problem
max
x,u
ˆ 1
0
≠u2 dt
subject to
x˙ = u+ ax, x(0) = 1, x(1) = 0, and u œ U = R.
Here, again, a œ R is a fixed but arbitrary constant.
Step 0: Since the problem is already in standard form, there is nothing to do.
Step 1: Notice that Step 1 doesn’t consider the boundary conditions. So Step 1
is the same as before, and we have that uˆ = p/2, again.
Step 2: Here is the first place things change. While the ODEs are the same, the
particular solutions we get will change because our boundary conditions and, thus,
our transversality condition has changed. Recall that our ODEs are
x˙ = p2 + ax and p˙ = ≠pa.