When I implement a Gauss-Jordan algorithm I can either have a 1 column result matrix or a multi-column result matrix (I mean the right hand side of the augmented matrix). The first case would be the one for linear systems of equations and the latter case would come into play when calculating the inverse of the source matrix.
Now I want to use full-pivoting with both row- and column-swapping. When I swap two rows in the source matrix I swap the same two rows in the result matrix as well.
What I don't get and couldn't find is how a column swap affects the result matrix. I can't really swap columns in the case of a single-column matrix. This would only work if the result matrix had the same amount of columns as the source matrix.
Could someone explain to me, how this is treated? I could only find examples for matrix inversion using partial pivoting, in which case this problem does not arise.
Best Answer
Row (Column) reducing a matrix is essentially a process whereby you do a series of elementary operations, which when taken together has the same effect as multiplying your equation (augmented system) by an invertible matrix (which is a product of elementary operations - row or column).
Now the distinction between a row operation and a column operation, is that a row operation is the same as multiplying on the left, eg $E_RA=E_RB$ whereas a column operation is the same as multiplying on the right, eg $AE_C=BE_C$. To be able to do any such an operation on an augmented system $[A : B]$ you must have that $A$ and $B$ have the same amount of rows (columns) for a row operation (column operation). So you cannot for example perform a column operation on an augmented system of $Ax=B$ where $A$ has multiple columns, and $B$ only one - within the context of solving the system for $x$ such an operation would be the same as trying to multiply $A$ and $B$ on the right with some elementary matrix - and the multiplication will be undefined for one of these matrices since they do not have the same amount of columns.
Now, just as you can row reduce the augmented system $[A : I]$ to $[I : A^{-1}]$ (if $A$ is invertible) you can also column reduce $\begin{bmatrix} A \\ \hline \\ I \end{bmatrix}$ to $\begin{bmatrix} I \\ \hline \\ A^{-1} \end{bmatrix}$. The use of augmenting with $I$ is that it allows you to retrieve the composition of elementary operations as a single invertible matrix.
What about both row and column operations on a matrix? Here the second observation I want to make, is that the whole point of augmenting and row or column reducing together, is that instead of multiplying one matrix and then the other with the same elementary matrix you do it all together in one step, but usually in the end (if you are not solving a linear system) you want to retrieve the nonsingular matrices composed of the series of elementary operations. So you want $P$ and/or $Q$ so that $PAQ$ is some meaningful equivalent matrix. Consider reducing
$\begin{bmatrix}A & I \\ I & 0 \end{bmatrix}$ to $\begin{bmatrix}B & P \\ Q & 0 \end{bmatrix}.$
(A needn't be square, but then the two identity submatrices are not the same size). What this means is that $PAQ=B$ and we say that $A$ and $B$ are equivalent.
If $A$ is invertible then you can also use both row and column operations:
Reduce $\begin{bmatrix}A & I \\ I & 0 \end{bmatrix}$ to $\begin{bmatrix}I & P \\ Q & 0 \end{bmatrix}.$
What this means is $PAQ=I$, so that $A^{-1}=QP$.
The one exception I need to mention where one can use both row and column operations on an augmented system $[A : I]$ is where $A$ is a symmetric and you reduce to $[D : Q^T]$ by doing a series of alternating column and row operations (column operation followed by exactly the same row operation, eg swap column 1 and 2 followed by swap row 1 and 2). Then in the end you will have $D=Q^TAQ$. Of course this only works because $A$ is symmetric.