$\newcommand{\G}{\mathcal{G}}
\newcommand{\K}{\mathcal{K}} $Question: When does $ \G $ admit a finite maximal closed subgroup?
Answer : Must be one of the following two cases
- $ \G $ is compact and simple
- $ \G $ is not compact in which case $ \G $ cannot be connected and moreover the component group $ \G/\G^\circ $ does not preserve any nontrivial proper closed subgroup (see comment from YCor about $ C_5 \ltimes \mathbb{R}^2 $).
From now on I will confine myself to the case that $ \G $ is connected.
In other words I will consider the statement "A connected Lie group $ \G $ has a finite maximal closed subgroup $ G $ if and only if $ \G $ is compact and simple."
The first implication is true.
Claim 1:
If a connected Lie group $ \G $ has a finite maximal closed subgroup $ G $ then $ \G $ must be compact and simple.
Proof:
Let $ \G $ be a connected Lie group and $ G $ a finite maximal closed subgroup. Since $ G $ is finite then $ G $ is a compact subgroup of $ \G $ so must be contained in a maximal compact subgroup, call it $ \K $. But $ G $ is a maximal closed subgroup thus we must have that $ \K=\G $ (note that $ \K $ cannot equal $ G $ since $ \K $ is connected (the maximal compact of a connected group is always connected)). So $ \G $ must be compact. If $ \G $ is not simple then there exists some morphism
$$
\pi: \G \to \G_i
$$
with positive dimensional kernel (here $ \G_i $ is basically one of the semisimple factors of $ \G $). Then
$$
\pi^{-1}(\pi(G))
$$
is a closed positive dimensional subgroup containing $ G $, contradicting the fact that $ G $ is a finite maximal closed subgroup. Thus if a connected Lie group $ \G $ has a finite maximal closed subgroup then we can conclude that $ \G $ is simple.
However the reverse implication does not hold: $ SU_{15} $ is an example of a compact connected simple Lie group with no finite maximal closed subgroups.
To see why this is the case it is important to note that
Claim 2: For a compact connected simple Lie group $ \G $, $ G $ is a finite maximal closed subgroup of $ \G $ if and only if $ G $ is Ad-irreducible and $ G $ is a maximal finite subgroup of $ \G $.
this follows from Corollary 3.5 of Sawicki and Karnas - Universality of single qudit gates.
Since a finite subgroup of $ SU_n $ is Ad-irreducible if and only if it is a unitary 2-design we have
Claim 3: $ G $ is a finite maximal closed subgroup of $ SU_n $ if and only if $ G $ is a maximal unitary 2-group in $ SU_n $.
By inspecting Theorem 3 of Bannai, Navarro, Rizo, and Pham Huu Tiep - Unitary $t$-groups one immediately determines that $ SU_{15} $ has no finite maximal closed subgroups.
Some of the main examples of finite maximal closed subgroups of $ SU_n $ include the normalizer in $ SU_{p^n} $ of an extra-special group $ p^{2n+1} $. Here $ p $ is an odd prime. There is also a similar construction $ p=2 $. These are known as (complex) Clifford groups. Then there are infinite families of examples relating to the Weil module for $ \operatorname{PSp}_{2n}(3) $ and another family related to $ U_n(2) $. Plus many exceptional cases.
A similar normalizer construction to the above gives finite maximal closed subgroups of all the $ \operatorname{SO}(2^n) $ as normalizers of an extra-special group $ 2^{2n+1} $. This is known as the real Clifford group. For details about real and complex Clifford groups see Nebe, Rains, and Sloane - Self-Dual Codes and Invariant Theory.
Best Answer
$\def\fg{\mathfrak{g}}\def\ft{\mathfrak{t}}\def\long{\text{long}}\DeclareMathOperator\SO{SO}\DeclareMathOperator\Sp{Sp}\DeclareMathOperator\SU{SU}$The point of this answer, as discussed in comments, is to note that this is NOT true in types BCFG. In comments, Mikhail Borovoi said he could prove the result is true in types ADE (now an answer), so collectively this resolves the claim.
Let $G$ be a compact connected simple Lie group with Lie algebra $\fg$. Choose a torus $T$ and with corresponding Lie algebra $\ft$, and let $$\fg = \ft \oplus \bigoplus_{\beta} \fg_{\pm \beta}$$ be the root decomposition, where $\beta$ runs over positive roots and each $\fg_{\pm \beta}$ is two dimensional. (Since I'm not tensoring by $\mathbb{C}$, I don't get to go all the way down to one dimensional subspaces.)
Now, suppose that $\fg$ has two lengths of roots, "long" and "short". A case by case check of the root systems shows that the sum of two long roots is always either $0$, long or not a root. Therefore, $$\fg_{\long} = \ft \oplus \bigoplus_{\beta\ \long} \fg_{\pm \beta}$$ is a subalgebra; let $G_{\long}$ be the corresponding subgroup.
Then $T \subsetneq G_{\long}$ and, because the only choice in defining $G_{\long}$ is the choice of $T$, the normalizer $N(T)$ also normalizes $G_{\long}$. So $G_{\long} N(T)$ is a subgroup containing $N(T)$ of dimension $\dim G_{\long} > \dim T$.
Explicitly, we have:
In type $B_n = \SO(2n+1)$, the group $G_{\long}$ is $\SO(2n)$ and $G_{\long} N(T) = O(2n)$.
In type $C_n = \Sp(2n)$, the group $G_{\long}$ is $\Sp(2)^n$ and I think that $G_{\long} N(T) = \Sp(2)^n \rtimes S_n$.
In type $F_4$, the group $G_{\long}$ is $\SO(8)$, or possibly one of the other compact real forms of $D_4$. I'm not sure what $G_{\long} N(T)$ is explicitly; it might depend on which compact form of $F_4$ we are using.
In $G_2$, the group $G_{\long}$ is $\SU(3)$. I think that $G_{\long} N(T)$ is $SU(3) \rtimes C_2$, with $C_2$ acting by an outer automorphism.
These examples, and a little bit of thought about the $F_4$ case, lead me to conjecture: $G_{\long} N(T)$ sits in a short exact sequence $1 \to G_{\long} \to G_{\long} N(T) \to \operatorname{Out}(G_{\long}) \to 1$. (I'm not ready to guess whether or not the sequence is always semidirect.)