My goal is to prove the following:
Let $G$ be a finite group and let $S \leq G$ be a simple subgroup. Suppose that $SH = HS$ for all subnormal subgroups $H$ of $G$. Show that $S$ is contained in the Wielandt subgroup $\omega(G)$ of $G$
Note: $\omega(G) = \bigcap N_G(H)$, where $H$ runs over the subnormal subgroups of $G$.
Here's my attempt:
We know (from this paper by Wielandt) that $\omega(G)$ contains every minimal normal subgroup of $G$ (as stated in the Encyclopedia of Math on the Wielandt subgroup). So it would suffice to prove that $S \subset \operatorname{Soc}(G)$, since this is the subgroup generated by such minimal normal subgroups.
Suppose, then, that this is not true. Then, we must have, by the simplicity of $S$, $S\cap \operatorname{Soc}(G) = 1$, which implies $S \cap M = 1$ for all minimal normal subgroups $M$ of $G$.
This feels like it can't happen, but I don't really know how to prove it. I haven't yet used $SH = HS$, so that's probably where to go next, but I just don't see where this hypothesis comes in… I know, from this post, that it wouldn't be possible if $S$ were nonabelian and subnormal, but I don't think it's necessarily the case here.
I don't actually need (or even really want) a complete solution, just a hint of what to do now, or if there is a better approach.
Thanks in advance!
Best Answer
What we will prove is the following for a single subnormal subgroup first.
Proposition Let $G$ be a finite group and let $S \leq G$ be a simple subgroup. Suppose that $H$ is a subnormal subgroup of $G$ with $SH = HS$. Then $S$ normalizes $H$.
Proof Note that from $SH=HS$ it does not follow immediately that $S$ normalizes $H$. What is true is that it implies that $SH$ is in fact a subgroup of $G$ and that is what we will use.
Since all groups are finite we can apply induction on the possible triples $(G,H,S)$ satisfying the conditions of the proposition. We choose $|G|$ as small as possible. If $|G|=1$ then the proposition is trivially true. Now consider the triple $(HS,H,S)$. This is a valid triple since $S \subseteq HS$ is simple, and $H \lhd \lhd HS$. Hence, if $|HS| \lt |G|$, then by induction, $S \subseteq N_{HS}(H) \subseteq N_G(H)$ and we are done.
We can assume that $G=HS$. Now $H \cap S \lhd \lhd S$, and since $S$ is simple we get, either $H \cap S=1$ or $H \cap S=S$. The latter case yields $S \subseteq H$, implying $G=HS=H$ and the statement becomes trivially true. So we need to deal with the case $H \cap S=1$. We can assume that $H$ is not normal, otherwise $S \subseteq N_G(H)=G$.
Hence, we can find in the normal series running from $H$ to $G$ a subgroup $H_1$ with $H \lhd H_1 \lhd \lhd G$, and $H \neq H_1$. By Dedekind's Modular Law $H(H_1 \cap S)=H_1 \cap HS= H_1 \cap G=H_1$. So $H_1 \neq H$ implies $H_1 \cap S \gt 1$, and since $S$ is simple and $H_1 \lhd \lhd G$, we conclude that $H_1 \cap S=S$, so $S \subseteq H_1$. But also $H \subseteq H_1$, whence $G=HS \subseteq H_1$, that is $G=H_1$. But then $H \lhd G$, a final contradiction.$\square$
Corollary Let $G$ be a finite group and let $S \leq G$ be a simple subgroup. Suppose that $SH = HS$ for all subnormal subgroups $H$ of $G$. Then $S$ is contained in the Wielandt subgroup $\omega(G)=\bigcap_{H \lhd \lhd G} N_G(H)$.
Note This is the solution to Problem 2A.4 from I.M. Isaacs, Finite Group Theory, p.54. See also here.
Added January 20th 2023. Here is a shorter proof, without induction that I learned from @mesel. Credits go to him!
Let $H \lhd \lhd G$, say $H=H_0 \lhd H_1 \lhd \cdots \lhd H_r=G$ be a normal series running from $H$ to $G$. We can assume that $S \nsubseteq H$, hence there is a minimal $i \in \{1,2, ..., r\}$ subject to $S \nsubseteq H_i$ and $S \subseteq H_{i+1}$. Note that since $S$ is simple, $H_i$ is subnormal, and $S \nsubseteq H_i$, we must have $S \cap H_i=1$.
Now we consider the normal closure of $H$ by $S$: $H^S=\langle s^{-1}hs: s \in S, h\in H\rangle$. Since $HS=SH$ this subgroup $H^S$ lies in $HS$. But $H_i \lhd H_{i+1}$, $S \subseteq H_{i+1}$, hence $S$ normalizes $H_i$, so $H^S \subseteq H_i^S=H_i$. We conclude that $H^S \subseteq HS \cap H_i=H(S \cap H_i)=H$ (here we applied Dedekind's Modular Law), that is $S \subseteq N_G(H)$. $\square$