I have the following problem:
Exercise 2.– A cup of coffee is initially at 100°C, and we want to lower its temperature to 0°C as quickly as possible by adding a fixed amount of milk. If $x(t)$ is the temperature of the coffee and milk mixture, the cooling law is given by
$$
x'(t) = -x(t) – 25u(t) – \frac{1}{4}x(t)u(t),$$
where $u(t)$ is what is being added of milk, restricted to $0 \leq u(t) \leq 1$ and also
$$
\int_0^T u(t)\,dt = 1,
$$
a) Reason that the optimal control should be of the form $$
u(t) = \begin{cases}
0, & 0 < t < t_0, \\
1, & t_0 < t < T,
\end{cases}
$$
for a certain $t_0$. b) Taking into account the previous section, find the optimal control.
Using a) it easy to compute $t_0$ and $T$, for which I obtained $t_0=0.697$ and $T=1.697$. But I don't know how to proceed with section a). I have the constraints $x(0)=100$ and $x(T)=0$, the Hamiltonian of the system is $$H=u+p(-x-25u-xu/4),$$ so the equation for $p$ is on the form $$p'=-H_x=p(1+u/4),$$ but since $u$ also depends on $t$ I don't know how to continue. Any help will be very appreciated. Thanks in advance!
EDITED: I managed to prove the following :
We define the Hamiltonian (H) for our system:
$$ H = \lambda(t) \left(-x(t) – 25u(t) – \frac{1}{4}x(t)u(t)\right) + \mu u(t), $$
where $\lambda(t)$ is the constant (or adjoint cost) multiplier and $\mu$ is the multiplier associated with the constraint $\int_0^T u(t) \, dt = 1$.
The maximum principle states that the optimal control $u^*(t)$ maximizes the Hamiltonian $H$ at each time instant $t$:
$$ u^*(t) = \arg\max_{0 \leq u \leq 1} H. $$
We calculate the derivative of the Hamiltonian with respect to $u$
$$ \frac{\partial H}{\partial u} = \lambda(t) \left(-25 – \frac{1}{4}x(t)\right) + \mu. $$
The decision of whether $u(t) = 0$ or $u(t) = 1$ at a given moment depends on the sign of the derivative
$$ \frac{\partial H}{\partial u} = \lambda(t) \left(-25 – \frac{1}{4}x(t)\right) + \mu $$
$$\frac{\partial H}{\partial u} > 0 \implies u^*(t) = 1$$
$$\frac{\partial H}{\partial u} < 0\implies u^*(t) = 0$$
I'm not really sure about the last two statements. However I don't know how to conclude the problem. Please help. Thanks in advance!
Best Answer
To start, note that once you know the optimal control is a bang-bang control with one switching time as described in part (a) you know how to find the exact formula for $u^*(t)$ and it is straightforward to to find $x^*(t)$ by plugging $u^*(t)$ into the differential equation and solving. So, the questions is how do we know that $u^*(t)$ necessarily has that form?
You've already made some good progress, which I'll add to here and give you a rough outline to fill in some more details yourself:
We transform this into the the new problem by setting $z(t) = \int_{0}^{t} u(\tau)\,d\tau$:
As you already did, we form the Hamiltonian $$H = 1 - \lambda\left(x + 25u + \frac{1}{4}xu\right) + \mu u$$
where the adjoints satisfy:
$$ \begin{align*} \lambda' &= -\frac{\partial H}{\partial x} = \lambda\left(1 + \frac{1}{4}u\right)\\[5pt] \mu' &= -\frac{\partial H}{\partial z} = 0 \implies \mu(t) = \text{const.} \end{align*} $$
Furthermore, we have
$$\frac{\partial H}{\partial u} = -\lambda\left(25 + \frac{1}{4}x\right) + \mu$$
and the conditions (as you've mostly established) that the sign of $\frac{\partial H}{\partial u}$ determines the optimal control. Additionally, we have the following condition on the Hamiltonian because the states are fixed at both endpoints:
$$ H\big\vert_{u = u^*,x = x^*, t = T} = 0 $$
Now, if we are aiming to show that $$u^*(t) = \begin{cases} 0, & 0 < t < t_0, \\ 1, & t_0 < t < T, \end{cases}$$ you can proceed by showing that $\frac{d}{dt}\frac{\partial H}{\partial u} < 0$ which will tell you that you have at most $1$ switching time. You can use extra condition on the Hamiltonian to show that $u^*(T) = 1$, which tells you that either $u^*(t) \equiv 1$ for all time or you have a bang-bang control as described in part (a). To rule out the first possibility you can plug in $u^*(t) \equiv 1$ to the differential equation for $x$ and you should find that you can't meet both boundary conditions, so the control has to take the other form and you can solve from there. Obviously there are some details to fill in, but I will leave those to you.