$A$ has the eigenbasis $\{| a \rangle \}$.
Let $\lambda$ be the eigenvalue of $B$ and $v \in V_{\lambda}$ an eigenvector ($Bv = \lambda v$).
Since $A$ and $B$ commute and are hermitian, following must be true:
$A^\dagger = A$, $B^\dagger = B$ and they are both diagonizable (source).
$$\Rightarrow Bv = \lambda v$$
$$\Rightarrow BAv = ABv = \lambda Av$$
$\rightarrow Av$ is also eigenvector of $B$ for the eigenvalue $\lambda$ and $Av \in V_{\lambda}$.
This is where I am stuck, I am not sure how to proceed and show that they must have the same eigenbasis but I think this must be the right trail.
Best Answer
Since $Av$ is an eigenvector of $B$, we can see that $A$ preserves the eigenspace $V_\lambda$ of $B$. We then restrict $A$ to $E_\lambda$, and then write $E_\lambda$ as a direct sum of eigenspaces for $A$ restricted to $E_\lambda$. Each of these eigenspaces is an eigenspace for both $A$ and $B$, so doing this over $A$ and $B$, we can write $\mathbb C^n$ as a sum subspaces, each of which is an eigenspace of $A$ and $B$, which gives an eigenbasis for both $A$ and $B$.