Hello I am trying to compute the derivative of the equation
$f = x^TAx $
I can rewrite it to
$f = \sum_{i,j} A_{ij}x_ix_j $
Then find its derivative
$\frac{df}{dx_k} = \sum_j A_{kj}x_j + \sum_i A_{ik}x_i $
How do I rewrite this in matrix form ?
[Math] Derivative – Convert summation to matrix
matricesmatrix-calculus
Related Solutions
Perhaps some help to compute the partial derivatives would be appreciated. It is always best to be explicit when one is a bit confused with heavy notation. Write $A = (a_{ij})$, $x = (x_1,\dots,x_n)^{\top}$ and $b = (b_1,\dots, b_m)^{\top}$, assuming $A$ is an $m \times n$ matrix. Then the $i^{\text{th}}$ component of $Ax-b$ is $$ (Ax-b)_i = \left( \sum_{j=1}^n a_{ij} x_j \right) - b_i $$ so that $$ (Ax-b)^{\top} (Ax-b) = \sum_{i=1}^m (Ax-b)_i^2 = \sum_{i=1}^m \left( \left( \sum_{j=1}^n a_{ij} x_j \right) - b_i \right)^2. $$ Suppose you want to compute the derivative with respect to $x_k$, $1 \le k \le n$ (I choose $k$ because choosing $i$ or $j$ would be confusing with the preceding subscripts used). Then $$ \frac{\partial}{\partial x_k} (Ax-b)^{\top} (Ax-b) = \sum_{i=1}^m \frac{\partial}{\partial x_k} \left( \left( \sum_{j=1}^n a_{ij} x_j \right) - b_i \right)^2 = \sum_{i=1}^m 2 \left( \left( \sum_{j=1}^n a_{ij}x_j \right) - b_i \right) (a_{ik}). $$ In particular, we can let $A_k = (a_{1k},a_{2k},\dots,a_{mk})$ so that $$ \frac{\partial}{\partial x_k} (Ax-b)^{\top} (Ax-b) = 2 \langle A_k, Ax-b \rangle $$ where $\langle - , - \rangle$ denotes inner product. You can use convexity arguments to show that any critical point is a minimizer in this case ; although you can see that the minimizer will not always be unique, even when $m=n$ ; it suffices for $A$ to be singular for this to happen.
Hope that helps,
$$\large{x=\begin{bmatrix} \\\end{bmatrix}_{n\times1}\to \\ x^TAx=\begin{bmatrix} \\\end{bmatrix}_{1\times n}\begin{bmatrix} \\\end{bmatrix}_{n\times n}\begin{bmatrix} \\\end{bmatrix}_{n\times1}=\begin{bmatrix} \\\end{bmatrix}_{1\times1}\to \\ \begin{bmatrix}x_1 & x_2 & ...&x_n \\\end{bmatrix}_{n\times1} \begin{bmatrix}a_{11} & a_{12} & ...&a_{1n}\\a_{21} &a_{22} &...&a_{2n}\\...\\a_{n1} &a_{n2}&...&a{nn} \\\end{bmatrix}_{n\times1} \begin{bmatrix}x_1 \\ x_2 \\ .\\.\\.\\x_n \\\end{bmatrix}_{n\times1}}$$ $$\begin{bmatrix}x_1a_{11}+x_2a_{21}+...+x_na_{n1} & x_1a_{12}+x_2a_{22}+...+x_na_{n2}& ...& x_1a_{1n}+x_2a_{2n}+...+x_na_{nn} \\\end{bmatrix}_{n\times1}\begin{bmatrix}x_1 \\ x_2 \\ .\\.\\.\\x_n \\\end{bmatrix}_{n\times1}=\\$$ $$=(x_1a_{11}+x_2a_{21}+...+x_na_{n1})x_1\\+(x_1a_{12}+x_2a_{22}+...+x_na_{n2})x_2\\+...\\+ (x_1a_{1n}+x_2a_{2n}+...+x_na_{nn})x_n\\=(\sum_{i=1}^nx_ia_{i1})x_1+(\sum_{i=1}^nx_ia_{i2})x_2+...+(\sum_{i=1}^nx_ia_{in})x_n=\\ \sum_{j=1}^n\sum_{i=1}^nx_ia_{ij}x_j $$
Best Answer
In index notation, you can write $$\frac{\partial f}{\partial x_k}=A_{kj}x_j + A_{ik}x_i$$ where a summation is implied by the presence of a repeated index.
You can also change a summation index (aka a dummy index), without altering the result, e.g. $(x_iy_i = x_ky_k)$.
So let's change both dummy indices to $p$ yielding $$\eqalign{ \frac{\partial f}{\partial x_k} &= A_{kp}x_p + A_{pk}x_p \cr &= (A_{kp}+A_{pk})\,x_p \cr }$$ which in matrix notation would be written as $\,(A+A^T)\,x$