The null space of $A$ is the set of solutions to $A{\bf x}={\bf 0}$. To find this, you may take the augmented matrix $[A|0]$ and row reduce to an echelon form. Note that every entry in the rightmost column of this matrix will always be 0 in the row reduction steps. So, we may as well just row reduce $A$, and when finding solutions to $A{\bf x}={\bf 0}$, just keep in mind that the missing column is all 0's.
Suppose after doing this, you obtain
$$
\left[\matrix{1&0&0&0&-1 \cr 0&0&1&1&0 \cr 0&0&0&0&0 \cr 0&0&0&0&0 \cr }\right]
$$
Now, look at the columns that do not contain any of the leading row entries. These columns correspond to the free variables of the solution set to $A{\bf x}={\bf 0}$ Note that at this point, we know the dimension of the null space is 3, since there are three free variables. That the null space has dimension 3 (and thus the solution set to $A{\bf x}={\bf 0}$ has three free variables) could have also been obtained by knowing that the dimension of the column space is 2 from the rank-nullity theorem.
The "free columns" in question are 2,4, and 5. We may assign any value to their corresponding variable.
So, we set $x_2=a$, $x_4=b$, and $x_5=c$, where $a$, $b$, and $c$ are arbitrary.
Now solve for $x_1$ and $x_3$:
The second row tells us $x_3=-x_4=-b$ and the first row tells us $x_1=x_5=c$.
So, the general solution to $A{\bf x}={\bf 0}$ is
$$
{\bf x}=\left[\matrix{c\cr a\cr -b\cr b\cr c}\right]
$$
Let's pause for a second. We know:
1) The null space of $A$ consists of all vectors of the form $\bf x $ above.
2) The dimension of the null space is 3.
3) We need three independent vectors for our basis for the null space.
So what we can do is take $\bf x$ and split it up as follows:
$$\eqalign{
{\bf x}=\left[\matrix{c\cr a\cr -b\cr b\cr c}\right]
&=\left[ \matrix{0\cr a\cr 0\cr 0\cr 0}\right]+
\left[\matrix{c\cr 0\cr 0\cr 0\cr c}\right]+
\left[\matrix{0\cr 0\cr -b\cr b\cr 0}\right]\cr
&=
a\left[ \matrix{0\cr1\cr0\cr 0\cr 0}\right]+
c\left[ \matrix{1\cr 0\cr 0\cr 0\cr 1}\right]+
b\left[ \matrix{0\cr 0\cr -1\cr 1\cr 0}\right]\cr
}
$$
Each of the column vectors above are in the null space of $A$. Moreover, they are independent. Thus, they form a basis.
I'm not sure that this answers your question. I did a bit of "hand waving" here. What I glossed over were the facts:
1)The columns of the echelon form of $A$ that do not contain leading row entries correspond to the "free variables" to $A{\bf x}={\bf 0}$. If the number of these columns is $r$, then the dimension of the null space is $r$ (again, if you know the dimension of the column space, you can see that the dimension of the null space must be the number of these columns from the rank-nullity theorem).
2) If you split up the general solution to $A{\bf x}={\bf 0}$ as done above, then these vectors will be independent (and span of course since you'll have $r$ of them).
In cases A and B, you can find the matrix of the linear transformation with respect to the canonical bases; in case A it is
$$
A=\begin{bmatrix}
2 & -1 \\
-8 & 4
\end{bmatrix}
$$
and in case B it is
$$
B=\begin{bmatrix}
4 & 1 & -2 & -3 \\
2 & 1 & 1 & -4 \\
6 & 0 & -9 & 9
\end{bmatrix}
$$
In general, when you have a linear transformation $T\colon\mathbb{R}^n\to\mathbb{R}^m$ and $\{e_1,e_2,\dots,e_n\}$ is the canonical basis of $\mathbb{R}^n$, you just write down (as columns), the vectors $T(e_1), T(e_2), \dots, T(e_n)$. A basis for the range can easily be computed by Gaussian elimination.
For case C, you don't have a "canonical basis", but you still can compute the matrix associated to the bases $\{1,x,x^2\}$ of $P_2$ (assuming it's the space of polynomials having degree at most 2) and $\{1,x,x^2,x^3\}$ of $P_3$. Since $T(1)=x=0\cdot1+1x+0x^2+0x^3$, $T(x)=x^2$, $T(x^2)=x^3$, the matrix is
$$
C=\begin{bmatrix}
0 & 0 & 0 \\
1 & 0 & 0 \\
0 & 1 & 0 \\
0 & 0 & 1
\end{bmatrix}
$$
The rank of this matrix is? And what can you conclude from this?
Best Answer
Your calculation for the basis of the kernel is correct.
Concerning your second question: Since the matrix has rank 3, you need three linearly independent column vectors of the matrix as a basis for the image. You can take the first three vectors, since they are linearly independent. Therefore, a basis of the image is
$$(1, 0, -1, -1), \qquad (0, 2, -1, 0), \qquad (0, 1, 0, 1).$$