Problem 1.

Consider the following optimal control problem

x^. = u, x(0) = 1

J = 1/2 ₀∫^Π (u² - x²)dt

(a) Explain whether or not this is a linear quadratic regulator problem

(b) Find the solution to this optimal control problem (u*(t) and x*(t))

Problem 2.

Consider a fixed final-time nonlinear optimal control problem

x^. = 2 sinu, x(0) = 0

-Π/2 ≤ u ≤ Π/2

J = ₀∫¹ cos² u(t) dt

(a) The final state x(1) is unconstrained (free). Find all optimal control u*(t) and the corresponding state x*(t)

(b) Now the terminal constraint x(1) = 1 is required. Find all solutions to the control u*(t) and state x*(t) that satisfies the Minimum Principle to this problem.

Problem 3.

Given the system and performance index as follows:

x. = x - u, x(0) = 1

J = 1/2₀∫^T (qx² + u²)dt

(a) For a finite T>0 and q = 1, this is a finite-time LQR problem. Find the solution to the differential Riccati equation for T = 1, and the optimal closed-loop control law for this problem.

(b) For T = +∞, verify that all the conditions required for an infinite-horizon LQR problem are met with any q > 0. Set q = 1, find the solution to the Algebraic Riccati Equation, the constant-gain optimal closed-loop control law, and the eigenvalue of the closed-loop system. Show details of the work

(c) When a constraint x(1) = 0 is imposed for T = 1, this is no longer an LQR problem (why?). Set q = 0, and find the solution (control and state) to this problem by applying directly the Minimum Principle.

Problem 4.

Consider the following optimal control problem:

x^.₁ = x₂
x^.₂ = u
x₁(0) = 0, x₂(0) = 0
x₂(1) = 1
1 - x₁(1) ≤ 0

J =  1/2₀∫^T u²(t)dt

Obviously the "non-standard" part in this problem is the inequality terminal constraint 1 - x₁(1) ≤ 0. In the absence of this constraint, the analytical solutions to the problem are available. Therefore it is possible to find the optimal solution of this problem by obtaining the analytical solution to optimal control problem and then solving a nonlinear programming problem, as described in the following.

(a) Ignore the constraint 1 - x₁(1) ≤ 0 for now. Apply the Minimum Principle to obtain the expressions of the optimal control u(t), and states x₁(t) and x₂(t) as functions of time. They should contain two unknown constants c₁ and c₂.

(b) By using the expressions of x₁(t), and x₂(t) in part (a), you can now express the terminal equality constraint x₂(1) = 1 as h(c₁,c₂)=0, and the terminal inequality constraint 1 - x₁(1) ≤ 0 as g(c₁,c₂)≤0, where h and g are linear functions of c₁ and c₂. Furthermore, using the u(t) in part (a) to obtain the analytical expression of J in terms of c₁ and c₂. Denote it by f(c₁, c₂), which should be a quadratic function of c₁ and c₂.

(c) Now the performance index and terminal (equality and inequality) constraints in the original optimal control problem are equivalent to a nonlinear programing problem (i. e., a quadratic programming problem) where the two unknowns c₁ and c₂ are found by

where is a positive definite matrix, and h and g are the two linear functions of c₁ and c₂ from part (a). The expressions of Q, h and g should all be well defined from the results in (a) and (b).

(d) Solve the problem in part (c), by hand, to obtain the final solution for the original optimal control problem x₁(t), x₂(t) and u(t). Show the details of the work.