$\DeclareMathOperator\SO{SO}\DeclareMathOperator\O{O}\DeclareMathOperator\SU{SU}\DeclareMathOperator\U{U}\DeclareMathOperator\S{S}\DeclareMathOperator\Sp{Sp}\DeclareMathOperator\PSL{PSL}\DeclareMathOperator\PSU{PSU}\newcommand{\irr}{\mathrm{irr}}\DeclareMathOperator\T{T}\DeclareMathOperator\A{A}\DeclareMathOperator\N{N}$Is the $6$-dimensional $ (2,0) $ irrep of $ \SU(3) $ maximal in $ \SU(6) $?
For those of you who are interested in context, I started wondering this the other day when I tried to write down the maximal subgroups of $ \SU(6) $. My guess so far is that the full list of maximal (proper closed) subgroups of $ \SU(6) $ is:
Type I (normalizer of maximal connected subgroup)
\begin{align*}
& \U(5) \cong \S(\U(5) \times \U(1)) \\
& \S(\U(4) \times \U(2)) \\
& \S(\U(3) \times \U(3))\rtimes \S_2 \\
& 6 \circ_2 \Sp(3) \\
& 6 \circ_2 \SO(6) \\
& 6 \circ_3 SU(3)_{\irr}
\end{align*}
Type II (finite maximal closed subgroup, for the last 2 groups GAP subscripts are used to label the center and the outer automorphisms when multiple groups of this structure description exist)
\begin{align*}
& 6.\A_7
\\
&6.\PSL(3,4).2_1
\\
&6_1.\PSU(4,3).2_2
\end{align*}
Type III (normalizer of a subgroup which is connected but not maximal connected)
\begin{align*}
& \N(\T^6)=\S(\U(1) \times \U(1) \times \U(1) \times \U(1) \times \U(1) \times \U(1)) \rtimes \S_6\\
&\S( \U(2) \times \U(2) \times \U(2) ) \rtimes \S_3\\
\end{align*}
Note on notation. $ \rtimes $ means split extension (semidirect product). $ \cdot $ means nonsplit extension. $ \circ $ denotes central product, in most cases here we have $ 6 \circ_2 H $, which is just the group generated by $ H $ and $ \zeta_6I $ but that group is not a direct product since already $ -I \in H $, we get a central product essentially with three $ H $ components. Similar idea for $ 6 \circ_3 \SU(3)_{\irr} $ having two components.
Here $ \N $ denotes normalizer. Recall that a positive dimensional (type I and type III above) maximal subgroup of a simple Lie group equals the full normalizer of its identity component.
The paper
classifies all maximal closed subgroups of $ \SU(n) $ whose identity component is not simple (here trivial counts as simple). According to table 5 the maximal closed subgroups of $ \SU(4) $ of this type are:
The normalizer of the maximal torus (row 4 table 5, $ \ell=6, p=1 $)
$$
\N(\T)=\S(\U(1) \times \U(1) \times \U(1) \times \U(1)) \rtimes \S_6
$$
and (row 4 of table 5, $ \ell=3, p=2 $)
$$
\S( \U(2) \times \U(2) \times \U(2) ) \rtimes \S_3
$$
As well as (row 1 table 5, $ p=5,q=1 $ )
$$
\S(\U(5) \times \U(1) )\cong \U(5)
$$
and (row 1 table 5, $ p=4,q=2 $ )
$$
\S(\U(4) \times \U(2) )
$$
and the normalizer of $ \S(\U(3) \times \U(3))= \{\begin{bmatrix} A & 0 \\ 0 & B \end{bmatrix}:A,B\in U(3),\det(A)\det(B)=1 \} $ which is a split extension (row 1 table 5 $ p=q=3 $)
$$
\langle \S(U(3) \times \U(3)),SWAP_{\oplus}\rangle \cong \S(\U(3) \times \U(3)) \rtimes \S_2
$$
where the normalizing matrix $ SWAP_{\oplus}=\begin{bmatrix} 0 & 0 & 0 & 0 & 0 & 1\\ 0 & 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 & 0 & 0 \\ 1 & 0 & 0 & 0 & 0 & 0 \end{bmatrix} $ swaps the two blocks in the direct sum.
Next, we consider maximal closed subgroups with nontrivial simple connected component.
By dimension, such a subgroup would be isogeneous to $ \SU(2)$, $\SU(3)$, $\Sp(2)$, $G_2$, $\SU(4)$, $\SO(7)$, $\Sp(3)$, $\SU(5)$, $\SO(8) $ of dimensions $ 3$, $8$, $10$, $14$, $21$, $21$, $24$, $28 $ respectively.
Of these the only one with 6d irreps are: 6d irrep of $ \SU(2) $, the $ (2,0) $ 6d irrep of $ \SU(3) $, fundamental irrep of $ \Sp(3) $,
Of these only
$$
6 \circ_2 \Sp(3)=\langle\zeta_6 I,\Sp(3)\rangle
$$
is maximal subgroup of $ \SU(6) $.
Even dimensional irreps of $ \SU(2) $ are always symplectic so all $ \SU(2) $ subgroups of $ \SU(6) $ are contained in a conjugate of $ \Sp(3) $. See this MathSE question.
Finally we consider subgroups with trivial connected component. These are finite since $ \SU(6) $ is compact. To be maximal they must at least be primitive. For example there is a very large $ 6 \circ_2 2.J_2 $ subgroup of $ SU(6) $ but it is not maximal because it is not even Lie primitive: it is contained in $ 6 \circ_2 \Sp(3) $. Also there is an $ \A_7 $ subgroup that is not Lie primitive, it is contained in $ \SO(6) $ since it is the standard $ \A_{n+1} $ subgroup of $ \SO(n) $ arising from the deleted permutation representation.
Even Lie primitive subgroups may not be maximal if they are contained in another larger Lie primitive finite subgroup. For example there is a subgroup $ 3.\A_7 \subset 6.\PSU(4,3) \subset \SU(6) $ which is Lie primitive but not maximal.
A maximal finite subgroup which is irreducible in the adjoint representation is always a maximal closed subgroup. This includes the following subgroups
The central product
$$
6.\A_7
$$
of order $ 6(2,520)=15,120 $ (maximal closed since it is maximal finite and a 2-design)
$$
6.\PSL(3,4).2_1
$$
of order $ 6(20,160)2 $ (maximal closed since it is maximal finite and a 3-design).
$$
6_1.\PSU(4,3).2_2
$$
of order $ 6(3,265,920)2 $ (maximal closed since it is maximal finite and a 3-design).
For references on designs and maximality see this MathSE question
This is consistent with the fact
that a maximal $ 2 $-design group is maximal closed ( all $ 3 $ designs are $ 2 $ designs).
This question cross posted from MSE
Best Answer
The Lie algebra of $SU(6)$ splits as the Lie algebra of $SU(3)$ plus a $27$-dimensional irreducible representation. (This follows from Pieri's rule: The adjoint representation is formed from the tensor product of the dual representation and the original representation, modulo the trivial subrepresentation. The dual has highest weight $(0,0,-2)$ so by Pieri's rule the tensor product is a sum of irreps with highest weights $(2,0,-2), (1,0,-1),(0,0,0)$. The $(0,0,0)$ is the trivial and the $(1,0,-1)$ is the adjoint so the remainder is irreducible.)
Thus the Lie algebra of any proper subgroup containing $SU(3)$ must be the Lie algebra of $SU(3)$, so any proper subgroup containing $SU(3)$ is the normalizer of $SU(3)$. But the centralizer of $SU(3)$ is just the scalars which are contained in the image of $SU(3)$ and the outer automorphism group of $SU(3)$ consists of a single element of order $2$ which does not fix this representation, so the centralizer is $SU(3)$.