Do matrix multiplication rules apply when multiplying matrices made up of smaller matrices

linear algebra

I came across a proof that did the following.

$a$, $b$, $c$ are $3 \times 1$ vectors. $A$ is a $3 \times 3$ matrix and $d$ is a $3 \times 1$ vector

$\begin{bmatrix}A & d\end{bmatrix} \begin{bmatrix} a & b & c \\ 0 & 0 & 1 \end{bmatrix} = \begin{bmatrix} Aa & Ab & Ac +d\end{bmatrix}$

So we are left multiplying a $4 \times 3$ matrix by a $3 \times 4$ matrix, but the elements of both matrices are themselves vectors and matrices.

I know this is correct because the rest of the proof in the paper follows. But, I don't really know how to justify using regular matrix multiplication properties to multiply two matrices made up of matrices. For example, $Aa$ is a matrix-vector product, whereas in regular matrix multiplication it would be some scalar product.

  1. Is there a name for this kind of linear algebra property & are there more like it?

  2. Why does it work? (Just a reference would suffice)

Best Answer

This block matrix notation is very useful. The meaning of the notation is clear: for example, if $$ A = \begin{bmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{bmatrix}, \quad B = \begin{bmatrix} b_{11} & b_{12} \\ b_{21} & b_{22} \end{bmatrix}, \quad C = \begin{bmatrix} c_{11} & c_{12} \\ c_{21} & c_{22} \end{bmatrix}, \quad D = \begin{bmatrix} d_{11} & d_{12} \\ d_{21} & d_{22} \end{bmatrix} $$ then $\begin{bmatrix} A & B \\ C & D \end{bmatrix}$ denotes the $4 \times 4$ matrix $$ \begin{bmatrix} A & B \\ C & D \end{bmatrix} = \begin{bmatrix} a_{11} & a_{12} & b_{11} & b_{12} \\ a_{21} & a_{22} & b_{21} & b_{22} \\ c_{11} & c_{12} & d_{11} & d_{12} \\ c_{21} & c_{22} & d_{21} & d_{22} \end{bmatrix}. $$ It is straightforward to prove the following basic rules for matrix multiplication using block notation:

  • $A \begin{bmatrix} B & C \end{bmatrix} = \begin{bmatrix} AB & AC \end{bmatrix}$.
  • $\begin{bmatrix} A \\ B \end{bmatrix} C = \begin{bmatrix} AC \\ BC \end{bmatrix}. $
  • $ \begin{bmatrix} A & B \end{bmatrix} \begin{bmatrix} C \\ D \end{bmatrix} = AC + BD. $

(For each rule, we must assume that the matrices $A, B, C$, and $D$ have compatible shapes.)

Using these basic rules, we can easily derive any more complicated block matrix multiplication rule that we need. For example,

\begin{align} \begin{bmatrix} A & B \\ C & D \end{bmatrix} \begin{bmatrix} X \\ Y \end{bmatrix} &= \begin{bmatrix} \begin{bmatrix} A & B \end{bmatrix} \begin{bmatrix} X \\ Y \end{bmatrix} \\ \begin{bmatrix} C & D \end{bmatrix} \begin{bmatrix} X \\ Y \end{bmatrix} \end{bmatrix} \\ &= \begin{bmatrix} AX + BY \\ CX + DY \end{bmatrix} \end{align} (assuming that the matrices $A, B, C, D, X$, and $Y$ have compatible shapes).

Related Question