wikipedia says that if the real function f has the Taylor expansion:
$f(x) = f(0) + f'(0) \cdot x + f''(0) \cdot \frac{x^2}{2!} + \dots$
then a matrix function can be defined by substituting x by a matrix: the powers become matrix powers, the additions become matrix sums and the multiplications become scaling operations.
I am not sure what is scaling operations here?
For function $f : \mathbf{R}^{n \times n} \rightarrow \mathbf{R}$, such as $f(\mathbf{X}) = trace(\mathbf{S} \mathbf{X})$, where $X$ is symmetric, what is the first order taylor expansion?
I got:
$\hat{f}(\mathbf{X} + \Delta) \approx trace(\mathbf{S} \mathbf{X}) + \mathbf{S} \Delta$
but $trace(\mathbf{S} \mathbf{X})$ is a scalar, while $\mathbf{S} \Delta$ is not, I must made some mistake here, any idea?
From the related literature, I can infer that someone got:
$\hat{f}(\mathbf{X} + \Delta) \approx trace(\mathbf{S} \mathbf{X}) + trace(\mathbf{S} \Delta)$
but I don't know how they got there, any help will be appreciated.
I just realized that matrix function refers to function which maps a matrix to another matrix. Sorry about that.
Here I just want to figure out the taylor expansion of a function maps a matrix to a scalar.
Best Answer
Let $f : \mathbb R^{n \times n} \to \mathbb R$ be defined by $f (\mathrm X) = \operatorname{tr} (\mathrm S \mathrm X)$. As the trace is a linear operator,
$$f (\mathrm X + h \mathrm V) = \operatorname{tr} (\mathrm S (\mathrm X + h \mathrm V)) = \operatorname{tr} (\mathrm S \mathrm X + h \mathrm S \mathrm V) = \operatorname{tr} (\mathrm S \mathrm X) + h \operatorname{tr} (\mathrm S \mathrm V) = f (\mathrm X) + h \operatorname{tr} (\mathrm S \mathrm V)$$
Hence, the directional derivative of $f$ in the direction of $\mathrm V$ is $D_{\mathrm V} f (\mathrm X) = \operatorname{tr} (\mathrm S \mathrm V)$.