Representation Theory – Can Solvable Connected Lie Groups Have Maximal Subgroups?

lie-groupsrt.representation-theory

Cross-posted from MSE.

Many interesting manifolds can be expressed as $ G/H $ for $ G $ a connected Lie group and $ H $ a maximal closed subgroup. Examples include the projective spaces $ \mathbb{C}P^n \cong \operatorname{SU}_n/U_{n-1} $ where $ U_{n-1} $ is maximal for $ n \geq 3 $, and $ \mathbb{R}P^n \cong \operatorname{SO}_n/O_{n-1} $, again $ O_{n-1} $ is maximal for $ n \geq 3 $. Another example is the Poincare homology sphere $ \operatorname{SO}_3(\mathbb{R})/A_5 $.

Solvmanifolds provide many interesting examples of manifolds, especially of torus bundles over tori (a solvmanifold is a manifold of the form $ G/H $ for $ G $ a solvable Lie group).

The examples I list above of manifolds $ G/H $, $ H $ maximal, all have the property that $ G $ is connected semisimple (indeed simple).

This leads me to wonder about the opposite case: maximal closed subgroups $ H $ of connected solvable Lie groups $ G $. Do they even exist?

Let $ G $ be a connected Lie group.

If $ G $ is abelian then certainly $ G $ does not have any maximal closed subgroups. Does the same hold for $ G $ solvable?

$\DeclareMathOperator\Ab{Ab}$Comment: Let $ G' $ be the commutator subgroup of the connected group $ G $. Let
$$
\Ab: G \to G/G'
$$

be the abelianization map. If $ H $ is a maximal closed subgroup of $ G $ then we must have
$$
\Ab(H)=G/G'
$$

because if $ \Ab(H) $ was properly contained then $ \Ab(H) $ would be a maximal closed subgroup of the connected abelian group $ G/G' $ which is impossible. In particular that implies that $ H $ does not contain $ G' $ (because if $ \Ab(H)=G/G' $
and $ H $ contained $ G' $ that would imply that $ H $ is all of $ G $, contradicting maximality).

Update: Recall that $ G $ is always a connected Lie group.

If $ G $ is nilpotent then there does not exist any maximal proper closed subgroup (proved in the original answer of LSpice).

If $ G $ is non-nilpotent then there does exist some maximal proper closed subgroup. We prove this with two cases.

$\DeclareMathOperator\Lie{Lie}$If the non-nilpotent group $ G $ is moreover non-solvable then we appeal to basically a Levi decomposition. $ \Lie(G) $ can be written as
$$
\Lie(G)= \mathfrak{g}_\text{solv} \rtimes \mathfrak{g}_\text{ss}.
$$

Let $ G_\text{solv} $ be a maximal solvable closed connected subgroup of $ G $ corresponding to the Lie subalgebra $ \mathfrak{g}_\text{solv} $. Let $ G_\text{ss} $ be a maximal semisimple closed connected subgroup of $ G $, corresponding to the Lie subalgebra $ \mathfrak{g}_\text{ss} $. Pick $ H_\text{max} $ to be a maximal proper closed subgroup of $ G_\text{ss} $ (there are lots of fairly well known maximal closed subgroups of semisimple groups). Then the group generated by $ G_\text{solv} $ and $ H_\text{max} $ should be roughly $ G_\text{solv} \rtimes H_\text{max} $ and should be a maximal proper closed subgroup of $ G $.

For the case that the non-nilpotent group $ G $ is solvable, then apply the answer from YCor (accepted below) which shows that a solvable non-nilpotent Lie group $ G $ must have a quotient which is one of the four solvable non-nilpotent subgroups of
$$
\operatorname{AGL}_1(\mathbb{C}) \cong \mathbb{C}^* \ltimes \mathbb{C}
$$

that YCor lists below. In that case there is a maximal proper closed subgroup of the quotient so we can pullback through the quotient map to get a maximal proper closed subgroup of the solvable non-nilpotent Lie group $ G $.

This proves the claim, from YCor's comment, that a connected Lie group $ G $ has a maximal proper closed subgroup if and only if $ G $ is non-nilpotent.

Best Answer

$\newcommand{\g}{\mathfrak{g}}$We can obtain the characterization by classifying just non-nilpotent connected Lie groups.

Let $G$ be a solvable, non-nilpotent connected Lie group. Suppose that every quotient Lie group of $G$ of dimension $<\dim(G)$ is nilpotent. Then $G$ has the form $H\ltimes V$ with both $H,V$ abelian of dimension $\le 2$ and $H$ acting nontrivially irreducibly on $V$ (see below for a list of possibilities).

First suppose that $G$ has a trivial center.

Let $\g$ be its Lie algebra, $(\g^i)_{i\ge 1}$ the lower central series and $\g^\infty=\bigcap_{i\ge 1}\g^i$, and $(G^i)$ the corresponding subgroups (a priori not closed). Note that $G^\infty$ is nilpotent (being contained in the derived subgroup $G^2$). Consider the adjoint representation of $G$. Since we can triangulate it (after complexification), we see that $\g^\infty$ maps to upper triangular matrices with zero diagonal. Hence $G^\infty$ maps to upper unipotent matrices. Thus $G^\infty$ is closed in $G$, and is simply connected. The action of $G$ on $G^\infty/[G^\infty,G^\infty]$ is not unipotent, since otherwise the action on $G^\infty$ would also be, and we would deduce that $G$ is nilpotent. So $G/[G^\infty,G^\infty]$ is a non-nilpotent Lie quotient of $G$, and hence, by the assumption, we deduce that $G^\infty$ is abelian. We claim that $\g^\infty$ is an irreducible module over $G/G^\infty$. Indeed, if it has a nonzero submodule, this corresponds to a connected nontrivial normal subgroup $N$ of $G$, contained in $G^\infty$. By the initial assumption, $G/N$ is nilpotent. But $G/G^\infty$ is the largest nilpotent quotient of $G$. So $N=G$. This proves irreducibility. Thus $G^\infty$ has dimension 1 or 2.

Let $\mathfrak{h}$ be a Cartan subalgebra in $\g$ (in the sense of Bourbaki: self-normalizing nilpotent subalgebra), and $H$ the corresponding subgroup (a priori possibly not closed). Then $\g=\mathfrak{h}+\g^\infty$. The intersection $\mathfrak{h}\cap\g^\infty$ is normalized by $\mathfrak{h}$, hence by $\mathfrak{g}$ (since $\g^\infty$ is abelian). By irreducibility, it is equal to either $\{0\}$ or $\g^\infty$. The latter is impossible, since $\mathfrak{h}$ is nilpotent. So $\g=\mathfrak{h}\ltimes\g^\infty$. By irreducibility, $[\mathfrak{h},\mathfrak{h}]$ centralizes $\g^\infty$. Let $\mathfrak{z}$ be the center of $\mathfrak{h}$. Then $[\mathfrak{h},\mathfrak{h}]\cap\mathfrak{z}$ is central in $\mathfrak{g}$, hence is zero. Since every nonzero ideal of a nilpotent Lie algebra meets the center, we deduce that $[\mathfrak{h},\mathfrak{h}]=0$: so $\mathfrak{h}$ is abelian. Also, the centralizer of $\g^\infty$ in $\mathfrak{h}$ is trivial, for the same reason.

Finally, we have one of the following

  1. $\dim(\g^\infty)=1$: then $\g=\mathbf{R}\ltimes\mathbf{R}$, so $G$ is the non-abelian $\mathbf{R}_{>0}\ltimes\mathbf{R}$.
  2. $\dim(\g^\infty)=2$, $\dim(\mathfrak{h})=1$. Then $G=\mathrm{U}(1)\ltimes\mathbf{C}$, or $G=\mathbf{R}\ltimes\mathbf{C}$ (acting by an unbounded 1-parameter subgroup of $\mathbf{C}^*$)
  3. $\dim(\g^\infty)=2$, $\dim(\mathfrak{h})=2$: then $G=\mathbf{C}^*\ltimes\mathbf{C}$.

If the center of $G$ is nontrivial, then it is discrete and the quotient has a trivial center. We obtain the possible connected covers of the previous examples (i.e., of $\mathrm{U}(1)\ltimes\mathbf{C}$ and of $\mathbf{C}^*\ltimes\mathbf{C}$).

(Note that $H$ is eventually closed: we see this by first viewing the cover $H\ltimes G^\infty$ of $G$, and then observing that this cover is trivial.)


Turning back to the issue of maximal subgroups: in all cases we have obtained a group of the form $H\ltimes V$ where $H$ acts irreducibly on $V$. So $H$ is a maximal proper subgroup, which is closed.

If $G$ is an arbitrary non-nilpotent connected Lie group, then it has a quotient of this form, and hence admits a maximal proper subgroup that is closed.