$A$ is a defective matrix, meaning that there are fewer linearly independent eigenvectors than eigenvalues; the algebraic multiplicity of $\lambda_1$ is $v_i = 2$ while the geometric multiplicity is $\mu_1 = 1$:
$$
A = \begin{bmatrix}1 & 1 \\ 0 & 1\end{bmatrix}, \lambda_1 = 1, e_1 = \begin{bmatrix}1 \\0\end{bmatrix}
$$
The block diagonal matrix $J$ (Jordan form) would be:
$$
J = \begin{bmatrix}\lambda_1 & 1 \\ 0 & \lambda_1\end{bmatrix}
$$
Correct me if I'm wrong:
- The number of blocks $J(\lambda_i)$ in $J$ equals the number of distinct eigenvalues ($1$ in this example). Each block is a square matrix of order $v_i$ (the algebraic multiplicity of $\lambda_i$). In this case there is only one block $J(\lambda_1)$ of order $2$.
- Each block $J(\lambda_i)$ has many miniblocks as the geometric multiplicity $\mu_i$ (in this case $2$), that is $J(\lambda_i, 1)…J(\lambda_i, \mu_i)$. Each miniblock has this strcture:
$$
\begin{bmatrix}\lambda_i & 1 \\ 0 & \lambda_i\end{bmatrix}
$$
Right now $J = \begin{bmatrix}1 & 1\\0 & 1\end{bmatrix}$. I need to find out the basis (the matrix $M$). Assume that the columns of $M$ are $x_1$ and $x_2$. We know that (Jordan form) $AM = MJ$. Computing the product a column at time we have:
$$
Ax_1 = \lambda_1x_1\\
x_1 = e_1 = \begin{bmatrix}1\\0\end{bmatrix}\\
Ax_2 = x_1 + \lambda_1x_2\\
(A – I\lambda_1)x_2 = x_1
$$
And $x_2$ is the generalized eigenvector ($\begin{bmatrix}c\\1\end{bmatrix}$).
Question 1: the choice of $c$ is up to me? I imagine that, because $M$ should be not singulare, I have to choose the right value by myself.
Question 2 is: is always possible to solve $(A – I\lambda_1)x_2 = x_1$ and why? In other words, the generalized eigenvector always exists?
Best Answer
If $A$ is $2 \times 2$, has eigenvalue $\lambda=1$ with algebraic multiplicity $2$ and geometric multiplicity $1$, then the null space (i.e. kernel) of $A-1I_2$ is 1-dimensional. In your example, it's the span of $\begin{bmatrix} 1 \\ 0 \end{bmatrix}$. The null space of $(A-1I_2)^2$ is 2-dimensional (and is all of $\mathbb{F}^2$ where $\mathbb{F}$ is whatever field of scalars you're working over -- probably $\mathbb{R}$ or $\mathbb{C}$). Notice that $(A-1I_2)\begin{bmatrix} a\\ b\end{bmatrix} = \begin{bmatrix} 0 & 1 \\ 0 & 0 \end{bmatrix} \begin{bmatrix} a\\ b \end{bmatrix} = \begin{bmatrix} b \\ 0 \end{bmatrix}$. So as long as $b=1$, you get $\begin{bmatrix} 1 \\ 0 \end{bmatrix}$. Thus $M=\begin{bmatrix} 1 & a \\ 0 & 1 \end{bmatrix}$ for any $a \in \mathbb{F}$ satisfies $M^{-1}AM=J$. So for question #1, "Yes" any scalar will do. This is analogous to the fact that when a matrix is diagonalizable it is diagonalized by infinitely many different matrices. For example: if $P^{-1}BP=D$ is diagonal, then replacing any column in $P$ by a non-zero multiple will also result in a matrix such that $Q^{-1}BQ=D$.
For question #2. The answer in general is "No". In your case and many "small" examples this works out. But in general if you have several Jordan blocks involving the same eigenvalue, this will fail.
Let me use an example to demonstrate. Let $C = \begin{bmatrix} 2 & 1 & 0 \\ 0 & 2 & 0 \\ 0 & 0 & 2 \end{bmatrix}$. The only eigenvalue is $\lambda=2$. It's algebraic multiplicity is 3 and geometric multiplicity is 2 [Note: The geometric multiplicity of an eigenvalue is equal to the number of Jordan blocks associated with that eigenvalue --- what you are calling Jordan miniblocks are actually the Jordan blocks --- each new Jordan block gives you one new (linearly independent) eigenvector.] Also, $C$ is already in Jordan form. It consists of 2 Jordan blocks: $J_1 = \begin{bmatrix} 2 & 1 \\ 0 & 2 \end{bmatrix}$ and $J_2 = [2]$.
The null space of $C-2I_3$ is 2-dimensional: $C-2I_3=\begin{bmatrix} 0 & 1 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{bmatrix}$. So the null space of $C-2I_3$ is spanned by $\begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}$ and $\begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix}$. The null space of $(C-2I_3)^2$ is 3-dimensional: $(C-2I_3)^2=0$ (this null space is all of $\mathbb{F}^3$). In summary, we have 2 (linearly independent) eigenvectors with eigenvalue 2 and 1 generalized eigenvector (which is not a regular eigenvector).
Suppose for some odd reason we came up with ${\bf v}_1 = \begin{bmatrix} 1 \\ 0 \\ 1 \end{bmatrix}$ and ${\bf v}_2 = \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix}$ as our basis of eigenvectors (this is indeed a basis -- although definitely not the "natural" choice). If we try to solve $(A-2I_3){\bf x} = {\bf v}_1$ or $(A-2I_3){\bf x} = {\bf v}_2$ to find our generalized eigenvector, we get stuck! Both of these systems are inconsistent: $\begin{bmatrix} 0 & 1 & 0 & : & 1 \\ 0 & 0 & 0 & : & 0 \\ 0 & 0 & 0 & : & 1 \end{bmatrix}$ and $\begin{bmatrix} 0 & 1 & 0 & : & 0 \\ 0 & 0 & 0 & : & 0 \\ 0 & 0 & 0 & : & 1 \end{bmatrix}$
This doesn't mean that $(A-2I_3)^2{\bf x} = {\bf 0}$ has no new solutions beyond ${\bf v}_1$ and ${\bf v}_2$. But it does mean that any new solutions cannot be obtained by "backing-up" from ${\bf v}_1$ or ${\bf v}_2$.
In general you cannot find a basis which puts your matrix into Jordan form by finding a basis of eigenvectors and "backing-up" chains.
To find your desired basis you must start with the longest chain, find the top of that chain, then find the top of the next longest chain (excluding what you just found) and continue working to the shortest chain.
In my example, I would find a basis for $\mathrm{Null}((A-2I_3)^1)$, say $\begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}$ and $\begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix}$. Then complete it to a basis for $\mathrm{Null}((A-2I_3)^2)$. The new vector that showed up say ${\bf w}_1 = \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix}$ would then belong to $\mathrm{Null}((A-2I_3)^2)$ but not to $\mathrm{Null}((A-2I_3)^1)$. So ${\bf w}_1$ is a generalized eigenvector but not an eigenvector and since it lives in $\mathrm{Null}((A-2I_3)^2)$ it is the start of a 2-chain. Let ${\bf w}_2 = (A-2I_3){\bf w}_1$ to complete this chain. I could then take ${\bf w}_2$ and complete it to a basis for $\mathrm{Null}((A-2I_3)^1)$. Call the new basis vector ${\bf w}_3$. Then $M = [{\bf w}_2 \; {\bf w}_1 \; {\bf w}_3]$ will put $C$ into its Jordan form [I reversed the order of my chain to make the $1$'s appear above (instead of below) the diagonal in the Jordan form].
A general procedure for finding the Jordan form works in a similar manner. Since it involves a lot of finding and completing bases, there can be a lot of variability in what matrix $M$ you end up with (as brought up in question #1).
Question #2b "Does a basis always exist which puts a matrix into Jordan form?" "Yes" if the characteristic polynomial factors into linear factors over your field of scalars (so always yes over the complex numbers). Why? Now that's complicated. I recommend "Spence, Insel, and Friedberg's Linear Algebra" for a reasonably easy readable treatment.