$$\left(\begin{array}{ccc|c}
\lambda & 1 & 1 & 1 \\
1 & \lambda & 1 & \lambda \\
1 & 1 & \lambda & \lambda^2
\end{array}\right)\sim
\left(\begin{array}{ccc|c}
\lambda+2 & \lambda+2 & \lambda+2 & 1+\lambda+\lambda^2 \\
1 & \lambda & 1 & \lambda \\
1 & 1 & \lambda & \lambda^2
\end{array}\right)\overset{(1)}\sim
\left(\begin{array}{ccc|c}
1 & 1 & 1 & \frac{1+\lambda+\lambda^2}{\lambda+2} \\
1 & \lambda & 1 & \lambda \\
1 & 1 & \lambda & \lambda^2
\end{array}\right)\sim
\left(\begin{array}{ccc|c}
1 & 1 & 1 & \frac{1+\lambda+\lambda^2}{\lambda+2} \\
0 & \lambda-1 & 0 & \frac{\lambda-1}{\lambda+2} \\
0 & 0 & \lambda-1 & \frac{\lambda^3+\lambda^2-\lambda-1}{\lambda+2}
\end{array}\right)\overset{(2)}\sim
\left(\begin{array}{ccc|c}
1 & 1 & 1 & \frac{1+\lambda+\lambda^2}{\lambda+2} \\
0 & 1 & 0 & \frac{1}{\lambda+2} \\
0 & 0 & 1 & \frac{(\lambda+1)^2}{\lambda+2}
\end{array}\right)\sim
\left(\begin{array}{ccc|c}
1 & 0 & 0 & \frac{-\lambda-1}{\lambda+2} \\
0 & 1 & 0 & \frac{1}{\lambda+2} \\
0 & 0 & 1 & \frac{(\lambda+1)^2}{\lambda+2}
\end{array}\right)\sim
$$
The step (1) is only valid for $\lambda\ne -2$.
The step (2) is only valid for $\lambda\ne 1$.
For values $\lambda\ne1,-2$ we get that there is only one solution $x_1=\frac{-\lambda-1}{\lambda+2}$, $x_2=\frac{1}{\lambda+2}$, $x_3=\frac{(\lambda+1)^2}{\lambda+2}$.
We can do the sanity check.
$\lambda x_1 + x_2 + x_3 = \frac{\lambda(-\lambda-1)+1+(\lambda+1)^2}{\lambda+2} = \frac{-\lambda^2-\lambda+1+\lambda^2+2\lambda+1}{\lambda+2} = \frac{\lambda+2}{\lambda+2} = 1$
$x_1 + \lambda x_2 + x_3 = \frac{(-\lambda-1)+\lambda+(\lambda+1)^2}{\lambda+2} = \frac{\lambda^2+2\lambda}{\lambda+2} = \lambda$
$x_1 + x_2 + \lambda x_3 = \frac{(-\lambda-1)+1+\lambda(\lambda+1)^2}{\lambda+2} = \frac{-\lambda+\lambda^3+2\lambda^2+\lambda}{\lambda+2} = \frac{\lambda^3+2\lambda^2}{\lambda+2}=\lambda^2$
Notice that the check works for $\lambda=1$. So we know that there is at least one solution for $\lambda=1$.
For $\lambda=1$
We get the system
$$\left(\begin{array}{ccc|c}
1 & 1 & 1 & 1 \\
1 & 1 & 1 & 1 \\
1 & 1 & 1 & 1
\end{array}\right)$$
which has solutions $x_1=1-s-t$, $x_2=s$, $x_3=t$ (where $s$, $t$ are arbitrary).
For $\lambda=-2$
$\left(\begin{array}{ccc|c}
-2 & 1 & 1 & 1 \\
1 & -2 & 1 & -2 \\
1 & 1 & -2 & 4
\end{array}\right)\sim
\left(\begin{array}{ccc|c}
-2 & 1 & 1 & 1 \\
1 & -2 & 1 & -2 \\
0 & 0 & 0 & 3
\end{array}\right)$.
So in this case there is no solution.
In short, row reduced echelon form(RREF) of a matrix $A$ is such that
i) Every leading entry is 1
ii) Any nonzero rows are above zero rows
iii) any leading entry is strictly to the right of any leading entries above that row
iv) any other entry in a column containing a leading entry is 0 except for the leading entry.
So it does not have to be put in augmented matrix $[A|b]$ to get a RRE form. You are comparing RRE form of matrix $A$ and $[A|b]$.
To see why the statement is true, suppose that you put the matrix $[A|b]$ into RRE form, so you have a matrix E. If E contains a leading entry in its last column, in terms of system of equations, what does it say? And what is the condition for E to not have any leading entry in last column?
Note: If RRE form of $[A|b]$ does contain a leading entry, then it is different from that of $A$. Also, note that RRE form of $[A|b]$ is m by n+1 whereas that of $A$ is m by n.
Solve:
$x+y=1$
$x+y=2$
Then we have
$\
A =
\left( {\begin{array}{cc}
1 & 1 \\
1 & 1
\end{array} } \right)
$
$\
b =
\left( {\begin{array}{cc}
1 \\
2
\end{array} } \right)
$
and $Ax=b$
If we turn A into RREF, we get
$\
E =
\left( {\begin{array}{cc}
1 & 1 \\
0 & 0
\end{array} } \right)
$
So A has rank 1
and if we put $[A|b]$ into RRE form, we get
$\
E' =
\left( {\begin{array}{cc}
1 & 1 & 0 \\
0 & 0 & 1
\end{array} } \right)
$
So augmented matrix has rank 2. Observe what last row says in terms of equations.
Best Answer
Let the system be $Ax=b$. If the coefficient matrix $A$ is $(m\times n)$ with $m>n$, then the augmented matrix must be checked, because it is still possible that $\operatorname{rk}[A\mid b]=n+1$. If $m\le n$, it is not necessary, because then $m=\operatorname{rk}A$ and $\operatorname{rk}[A\mid b]\le \min(m,n+1)=m$.