I was trying to find the Frechet derivative of $f = \exp(X)$, where $X \in \mathbb{R}^{n\times n}$ is positive definite. I thought it ought to be $\exp(X)$.
I see results where the derivative is with respect to a scalar argument, but this question has not been asked before.
I tried to see if I could find $Df_X$ starting with
$Df_X[H] = \exp(X+H) – \exp(X)$. If I can show that the right hand side evaluates to $I + XH + X^2H/2 + \ldots = \exp(X)H$, I am done.
After I use the series definition, however, I am lost because I see no reason to assume that $A$ and $H$ commute.
Please help.
EDIT
Following the suggestion in the comment, I try to compute the Gateaux derivative as $\exp(X + tH)$ by writing down the first few terms.
$\exp(X+tH) = I + (X + tH) + (X^2 + tXH + tHX + t^2H^2)/2 + \cdots$
$\dfrac{d}{dt}\exp(X+tH)\Big|_{t=0} = H + (XH+HX)/2 + \cdots$
And now am stuck again. It seems the expression on the right cannot be rearranged to give what I want.
I think it is the derivative of the trace of the exponential, not the exponential itself that yields $\exp(X)$
Best Answer
Let $f : \mathbb R^{n \times n} \to \mathbb R^{n \times n}$ be defined by $f (X) = \exp(X)$. Hence,
$\begin{array}{rl} f (X + h V) &= \exp(X + h V)\\ &= I_n + (X + h V) + \frac{1}{2!} (X + h V)^2 + \frac{1}{3!} (X + h V)^3 + \cdots\\ &= I_n + X + h V + \frac{1}{2!} X^2 + \frac{h}{2!} (X V + V X) + \frac{h^2}{2!} V^2 + \frac{1}{3!} X^3 + \\ &\,\,\,\,\,+ \frac{h}{3!} (X^2 V + X V X + V X^2) + \frac{h^2}{3!} (X V^2 + V X V + V^2 X) + \frac{h^3}{3!} V^3 + \cdots\\ &= f (X) + h \left( V + \frac{1}{2!} (X V + V X) + \frac{1}{3!} (X^2 V + X V X + V X^2) + \cdots\right) + \cdots\end{array}$
Thus, the directional derivative of $f$ in the direction of $V$ at $X$ is given by
$$\begin{array}{rl} D_V f (X) &= \displaystyle\lim_{h \to 0} \frac{1}{h} \left( f (X + h V) - f (X) \right)\\ &= V + \frac{1}{2!} (X V + V X) + \frac{1}{3!} (X^2 V + X V X + V X^2) + \cdots\end{array}$$
We then write
$$D_V f (X) = M_0 + \frac{1}{2!} M_1 + \frac{1}{3!} M_2 + \frac{1}{4!} M_3 + \cdots$$
where
$$\begin{array}{rl} M_0 &= V\\ M_1 &= X V + V X =: \{X,V\}\\ M_2 &= X^2 V + X V X + V X^2\\ &\vdots\\ M_k &= \displaystyle\sum_{i=0}^k X^{k-i} V X^i\end{array}$$