The Frobenius norm of $A$ is the squareroot of the sum of all of the eigenvalues (the trace) of $A^*A$, and since all eigenvalues of $A^*A$ are nonnegative, it follows that the largest eigenvalue is less than or equal to the sum of all of the eigenvalues.
If $p=2$ it's the Frobenius norm, right?
No, if $p=2$ that is another characterization of the spectral norm. A proof of this is sketched in Michalis's answer.
Put $B=A^*A$ which is a Hermitian matrix. A linear transformation of the Euclidean vector space $E$ is Hermite iff there exists an orthonormal basis of E consisting of all the eigenvectors of $B$. Let $\lambda_1,...,\lambda_n$ be the eigenvalues of $B$ and $\left \{ e_1,...e_n \right \}$ be an orthonormal basis of $E$. Denote by $\lambda_{j_{0}}$ to be the largest eigenvalue of $B$.
For $x=a_1e_1+...+a_ne_n$, we have $\left \| x \right \|=\left \langle \sum_{i=1}^{n}a_ie_i,\sum_{i=1}^{n}a_ie_i \right \rangle^{1/2} =\sqrt{\sum_{i=1}^{n}a_i^{2}}$ and
$Bx=B\left ( \sum_{i=1}^{n}a_ie_i \right )=\sum_{i=1}^{n}a_iB(e_i)=\sum_{i=1}^{n}\lambda_ia_ie_i$. Therefore:
$\left \| Ax \right \|=\sqrt{\left \langle Ax,Ax \right \rangle}=\sqrt{\left \langle x,A^*Ax \right \rangle}=\sqrt{\left \langle x,Bx \right \rangle}=\sqrt{\left \langle \sum_{i=1}^{n}a_ie_i,\sum_{i=1}^{n}\lambda_ia_ie_i \right \rangle}=\sqrt{\sum_{i=1}^{n}a_i\overline{\lambda_ia_i}} \leq \underset{1\leq j\leq n}{\max}\sqrt{\left |\lambda_j \right |} \times (\left \| x \right \|)$
So, if $\left \| A \right \|$ = $\max \left\{ \|Ax\| : \|x\| = 1 \right\}$ then $\left \| A \right \|\leq \underset{1\leq j\leq n}\max\sqrt{\left |\lambda_j \right |}$. (1)
Consider: $x_0=e_{j_{0}}$ $\Rightarrow \left \| x_0 \right \|=1$ so that $\left \| A \right \|^2 \geq \left \langle x_0,Bx_0 \right \rangle=\left \langle e_{j_0},B(e_{j_0}) \right \rangle=\left \langle e_{j_0},\lambda_{j_0} e_{j_0} \right \rangle = \lambda_{j_0}$. (2)
Combining (1) and (2) gives us $\left \| A \right \|= \underset{1\leq j\leq n}{\max}\sqrt{\left | \lambda_{j} \right |}$ where $\lambda_j$ is the eigenvalue of $B=A^*A$
Conclusion: $$\left \| A \right \| _2=\sqrt{\lambda_{\text{max}}(A^{^*}A)}=\sigma_{\text{max}}(A)$$
Best Answer
For any square $A$, $\rho(A)\leq\|A\|_2$, where $\rho(A)$ is the spectral radius of $A$, with the equality (but not necessarily) if $A$ is normal. Besides the general inequality, $\rho(A)$ and $\|A\|_2$ can be completely unrelated. Consider, e.g., $$ A_\alpha:=\pmatrix{0&\alpha\\0&0} $$ with $\rho(A_\alpha)=0$ but $\|A_\alpha\|_2=|\alpha|$. All the eigenvalues are zero but the 2-norm can be an arbitrary non-negative number (depending on $\alpha$).