Can solvable Lie groups have maximal subgroups

lie-groupsrepresentation-theory

Many interesting manifolds can be expressed as $ G/H $ for $ G $ a Lie group and $ H $ a maximal closed subgroup. Examples include the projective spaces $ \mathbb{C}P^n \cong SU_n/U_{n-1} $ where $ U_{n-1} $ is maximal for $ n \geq 3 $, and $ \mathbb{R}P^n \cong SO_n/O_{n-1} $, again $ O_{n-1} $ is maximal for $ n \geq 3 $. Another example is the Poincare homology sphere $ SO_3(\mathbb{R})/A_5 $.

Solvmanifolds provide many interesting examples of manifolds, especially of torus bundles over tori (a solvmanifold is a manifold of the form $ G/H $ for $ G $ a solvable Lie group).

The examples I list above for manifolds $ G/H $, $ H $ maximal, all the have the property that $ G $ is semisimple (indeed simple).

This leads me to wonder about maximal closed subgroups $ H $ of solvable Lie groups $ G $. Do they even exist?

Let $ G $ be a connected Lie group.

If $ G $ is abelian then certainly $ G $ does not have any maximal closed subgroups. Does the same hold for $ G $ solvable?

comment (inspired by the comment from
Eero Hakavuori): Let $ G' $ be the commutator subgroup of $ G $. Let
$$
Ab: G \to G/G'
$$

be the abelianization map. If $ H $ is a maximal closed subgroup of $ G $ then we must have
$$
Ab(H)=G/G'
$$

because if $ Ab(H) $ was properly contained then $ Ab(H) $ would be a maximal closed subgroup of the connected abelian group $ G/G' $ which is impossible. In particular that implies that $ H $ does not contain $ G' $ (because if $ Ab(H)=G/G' $
and $ H $ contained $ G' $ that would imply that $ H $ is all of $ G $, contradicting maximality).

Update: A thorough answer is given by Ycor in the cross post to MO https://mathoverflow.net/questions/433142/can-solvable-connected-lie-groups-have-maximal-subgroups/433144?noredirect=1#comment1115468_433144 a second answer by LSpice, supplementing his first answer, can also be found there.

Update: Recall that $ G $ is always a connected Lie group.

If $ G $ is nilpotent then there does not exist any maximal proper closed subgroup (proved in the original answer of LSpice).

If $ G $ is non-nilpotent then there does exist some maximal proper closed subgroup. We prove this with two cases.

If the non-nilpotent group $ G $ is moreover non-solvable then we appeal to basically a Levi decomposition. $ Lie(G) $ can be written as
$$
Lie(G)= \mathfrak{g}_{solv} \rtimes \mathfrak{g}_{ss}
$$

Let $ G_{solv} $ be a maximal solvable closed connected subgroup of $ G $ corresponding to the Lie subalgebra $ \mathfrak{g}_{solv} $. Let $ G_{ss} $ be a maximal semisimple closed connected subgroup of $ G $, corresponding to the Lie subalgebra $ \mathfrak{g}_{ss} $. Pick $ H_{max} $ to be a maximal proper closed subgroup of $ G_{ss} $ (there are lots of fairly well known maximal closed subgroups of semisimple groups). Then the group generated by $ G_{solv} $ and $ H_{max} $ should be roughly $ G_{solv} \rtimes H_{max} $ and should be a maximal proper closed subgroup of $ G $.

For the case that the non-nilpotent group $ G $ is solvable, then apply the answer from Ycor (accepted below) which shows that a solvable non-nilpotent Lie group $ G $ must have a quotient which is one of the four solvable non-nilpotent subgroups of
$$
AGL_1(\mathbb{C}) \cong \mathbb{C}^* \ltimes \mathbb{C}
$$

that Ycor lists below. In that case there is a maximal proper closed subgroup of the quotient so we can pullback through the quotient map to get a maximal proper closed subgroup of the solvable non-nilpotent Lie group $ G $.

This proves the claim, from Ycor's comment, that a connected Lie group $ G $ has a maximal proper closed subgroup if and only if $ G $ is non-nilpotent.

Best Answer

I am not sure about the etiquette for answering cross-posted questions. I also posted this answer on MO. If it is not appropriate to post it here, then I am happy to delete it. See also my answer on MO discussing certain special soluble groups, and @YCor's more general answer on MO that handles the general soluble case.

A connected, soluble nilpotent Lie group $G$ has no maximal proper, closed subgroup. (Thanks to @YCor for pointing out on MathOverflow, with a counterexample, that my original claim was incorrect.)

We proceed by induction on the dimension of $G$. If the dimension is $0$, then $G$ is trivial, and we are done; so suppose that the dimension is positive, and hence that $\operatorname Z(G)^\circ$ is a positive-dimensional subgroup of $G$.

Suppose that $H$ is a maximal proper, closed subgroup of $G$. If $H$ contains $\operatorname Z(G)^\circ$, then $H/{\operatorname Z(G)}^\circ$ is a maximal proper, closed subgroup of $G/{\operatorname Z(G)}^\circ$, which is a contradiction. Therefore, $H$ is a proper subgroup of $H\cdot\operatorname Z(G)^\circ$. By maximality, we have that $H\cdot\operatorname Z(G)^\circ$ is dense in $G$. (It seems plausible that $H\cdot\operatorname Z(G)^\circ$ is already closed, but I do not know how to prove it.) Thus, for every pair of elements $g, g' \in G$, we have sequences $((h_n, z_n))_n$ and $((h_n', z_n'))_n$ in $H \times \operatorname Z(G)^\circ$ whose images under the multiplication map $H \times \operatorname Z(G)^\circ \to G$ converge to $g$ and $g'$. Then the sequence $([h_n z_n, h_n'z_n^{\prime\,{-1}}])_n$ equals $([h_n, h_n'])_n$, hence lies in $H$; and converges to $[g, g']$, which therefore belongs to $H$. Then $H$ contains the derived subgroup of $G$, which, as you have observed, is a contradiction.

Related Question