[Math] How to convert index notation equations to matrix/tensor equations

linear algebratensors

In many areas within computer science, one often arrives at an equation that uses index notation on some scalar elements of a vector/matrix/tensor, for example:
$$
a_i^{(s)} = \sum_j \frac{a_j^{(s+1)} \cdot h_{ij}^{(s)} \cdot b_i^{(s)}}{\sum_k h_{kj}^{(s)} \cdot b_{k}^{(s)}}
$$
where $a_i^{(s)}$, $a_i^{(s+1)}$, $b_i^{(s)}$ are elements of vectors $\bf{a^{(s)}}$, $\bf{a}^{(s+1)}$, and $\bf{b^{(s)}}$, and $h_{ij}^{(s)}$ is an element of a matrix $H^{(s)}$.
Then, it is often desirable to convert such an equation to its tensor form, in this case (making some assumptions about the compatibility of dimensions):
$$
\mathbf{a}^{(s)} = \mathbf{b}^{(s)} \odot (H^{(s)}. \Big( \mathbf{a}^{(s+1)} \oslash (H^{(s)T} \cdot \mathbf{b}^{(s)})\Big))
$$
Where $\oslash$ and $\odot$ are Hadamard division and product respectively.

However, converting such an equation in this way seems to take a lot of guesswork and effort (even though I do not consider myself foreign to linear algebra), whilst one may often encounter even more complicated equations with more than two dimensions.

My question then is: is there some sort of method for dealing with these sort of equations, or even a way of practicing to be able to do this quickly?

Best Answer

Examples:

$c = x ^ { \top } y \quad c = x _ { i } y ^ { i }$

$x = A y \quad x ^ { i } = A _ { j } ^ { i } y ^ { j }$

$x ^ { \top } = y ^ { \top } A \quad x _ { j } = y _ { i } A _ { j } ^ { i }$

$C = A \cdot B \quad C _ { k } ^ { i } = A _ { j } ^ { i } B _ { k } ^ { j }$

$A = x y ^ { \top } \quad A _ { j } ^ { i } = x ^ { i } y _ { j }$

$z = x \odot y \quad z ^ { i } = x ^ { i } y ^ { i }$

$z = x \oslash y \quad z ^ { i } = x ^ { i }/ y ^ { i }$

$A=x\otimes y \quad A^{ij}=x^i y^j$

$C=A\otimes B\quad C^{ij}_{kl}=A_k^i B_l^j$

$B=A\otimes x\quad B_{ij}^k=A_{ij}x^k$

$B = A \operatorname { diag } ( x ) \quad B _ { j } ^ { i } = A _ { j } ^ { i } x _ { j }$

$B = \operatorname { diag } ( x ) A \quad B _ { j } ^ { i } = x ^ { i } A _ { j } ^ { i }$

The formulas in the above is based on Einstein Notation(Abstract Index Notation), more information can be found here: https://zhangwfjh.wordpress.com/2014/07/19/einstein-notation-and-generalized-kronecker-symbol/

Related Question