I am reading a very nice paper of Newton and Thorne, Symmetric power functoriality for holomorphic modular forms, and there is an argument concerning the (Zariski-closure of) image of certain $p$-adic Galois representations that I do not fully understand. In case it is helpful, I am talking about Lemma 3.5 of the paper, but I doubt that the context is too relevant — I think I have understood the necessary $p$-adic Hodge theory arguments, and the only thing left is something routine with algebraic groups.
Let $\rho: G_{\mathbf{Q}_p} \to \mathrm{GL}_2(\overline{\mathbf{Q}}_p)$ be a continuous irreducible representation. Then (essentially by definition of what a reductive group is), the Zariski closure of the image of $\rho$ (call it $H$) is a reductive subgroup of $\mathrm{GL}_2$. From the context of the paper ($\rho$ comes from a classical modular form of weight $k \geq 2$), $\rho$ is Hodge–Tate with distinct Hodge–Tate weights. In particular, the Sen operator is regular semisimple, so a theorem of Sen implies that $H$ has Lie algebra base-changing to $\mathbf{C}_p$ to something containing a regular semisimple element. This in turn implies that $H$ is NOT a finite group, as its Lie algebra is nontrivial (could be convenient if the approach ends up being to use some classification theorem for all algebraic subgroups of $\mathrm{GL}_2$ but I was hoping to avoid that route and understand something more conceptual). It also implies that the maximal torus of $H$ is a regular torus in $\mathrm{GL}_2$.
Somebody told me that we could do a $p$-adic Hodge theory argument to argue that $H$ is connected, but in fact since the modular form $\rho$ comes from could have $p$ in the level, there is no crystalline assumption to be had, and I don't think that we can actually do that.
Anyhow, it seems that using just this information ($H$ an infinite reductive subgroup of $\mathrm{GL}_2$ whose maximal torus is regular in $\mathrm{GL}_2$), it is supposed to follow that if $H$ does not contain $\mathrm{SL}_2$, then it is contained in the normalizer of a maximal torus of $\mathrm{GL}_2$. Perhaps I could get this out of the general classification of algebraic subgroups of $\mathrm{GL}_2$ (by hopefully ruling out a bunch of cases due to not being reductive), which I don't know how to prove but was able to find on Google. Is this the way to go? Or is there a more conceptual way of proving what is used here? It would be nice, of course, to have something that generalizes to higher-dimensional representations.
Best Answer
Let $T_H$ be a maximal torus in $H$.
Suppose that $H^\circ$ is not a torus. Then $T_H$ has non-trivial roots on $\operatorname{Lie}(H)$, and the subgroup of $H$ generated by opposite root groups is $\operatorname{SL}_2$.
Thus, if $H$ does not contain $\operatorname{SL}_2$, then its identity component equals $T_H$; and you have indicated that that torus is regular in $G = \operatorname{GL}_2$, which I take to mean that its centraliser $T_G \mathrel{:=} C_G(T_H)$ in $G$ is a maximal torus in $G$. Thus, $H$ normalises the maximal torus $T_G = C_G(H^\circ)$ in $G$.