I came up with a "proof" for the Maschke's Theorem in the representation theory of finite groups that seems to make sense. But it doesn't use the fact that the group being represented is finite. So I suspect there must be something wrong with it. Can someone please help me find where it goes wrong?
In the following, the underlying field is assumed to have characteristic $0$, and all the vector spaces involved are assumed to be finite dimensional.
The following form of the theorem is taken from Fulton&Harris' Representation Theory – A First Course (Proposition 1.5).
Maschke's Theorem. If $W$ is a subrepresentation of a representation $V$ of a finite group $G$, then there is a complementary invariant subspace $W'$ of $V$, so that $V=W\oplus W'$.
Proof. Firstly, since the action of every $g\in G$ is invertible, if a subspace $V'\subset V$ is $g$-invariant, then $g.V'=V'$ without loss of dimensions. Hence $V'$ is also $g^{-1}$-invariant and $g^{-1}.V'=V'$.
Suppose for now $W\subset V$ is an arbitrary subspace, not necessarily invariant under the actions of any $g\in G$. Let $\pi: V\to W$ be the corresponding linear projection, which induces a direct sum decomposition $V=W\oplus \text{Ker}(\pi)$.
Define $p_{g,\pi}:=g^{-1}\pi g$. Then it is easy to verify $p_{g,\pi}^2=g^{-1}\pi gg^{-1}\pi g=p_{g,\pi}$ and so it is also a linear projection. This would induce an alternative direct sum decomposition $V= \text{ran}(p_{g,\pi})\oplus \text{Ker}(p_{g,\pi})$. We further have $g. \text{ran}(p_{g,\pi})\subset W$. In fact, for every $x\in \text{ran}(p_{g,\pi})$, we have $x=p_{g,\pi}(x)=g^{-1}\pi gx$ and so $gx=\pi gx\in W$.
In case $W$ is $g$-invariant, by first paragraph, $W$ is also $g^{-1}$-invariant. Therefore, since $\pi|_W$ is the identity map, for every $w\in W$, we have $\text{ran}(p_{g,\pi})\ni p_{g,\pi}(w)=g^{-1}\pi gw=g^{-1}gw=w$. Hence $W\subset \text{ran}(p_{g,\pi})$.
Then by both previous paragraphs, $g. \text{ran}(p_{g,\pi})\subset W\subset \text{ran}(p_{g,\pi})$ and so $ W= \text{ran}(p_{g,\pi})$, since $g$ is invertible. This gives $\text{Ker}(p_{g,\pi})=\text{Ker}(\pi)$ and $p_{g,\pi}=\pi$. That is, the two direct sum decompositions of $V$ become identical.
Then, we can check that $g$ and $g^{-1}$ both preserve the two identical kernels. In fact, for every $y\in \text{Ker}(p_{g,\pi})=\text{Ker}(\pi)$, we have $p_{g,\pi}(g^{-1}y)=g^{-1}\pi gg^{-1}y=g^{-1}\pi y=0$, and so $g^{-1}y\in \text{Ker}(p_{g,\pi})=\text{Ker}(\pi)$. By the invertibility of $g^{-1}$, the two identical kernels are both $g$-invariant and $g^{-1}$-invariant.
Lastly, if $W$ is $G$-invariant, then every $g\in G$ induces the same projection $p_{g,\pi}=\pi$ as reasoned above, and the subspace $W':=\text{Ker}(\pi)$ is then also $G$-invariant and complementary to $W$ such that $V=W\oplus W'$. ${\rm \square}$
As you can see, nowhere in the "proof" above is the fact used that $G$ is a finite group, and that's horrifying to me…
UPDATE:
According to Eric's answer, the issue is that, once we have $W= \text{ran}(p_{g,\pi})$, we cannot conclude the two projections are identical because their kernels might still be different. However, we would still have direct sum decomposition $V=W\oplus \text{Ker}(p_{g,\pi})$. This would be true for every $g\in G$. But the corresponding kernel would change as $g$ runs through $G$. This is where the "averaging technique" comes into play, and how the finiteness of $G$ plays a key role.
Best Answer
You seem to be assuming that there is a unique projection onto a subspace of a vector space. That is not true. For instance, if $W$ is two-dimensional space with basis $\{e_1,e_2\}$, then for any scalar $c$, $\{e_1,e_2+ce_1\}$ is also a basis, and there is a projection onto the span of $e_1$ corresponding to this basis. These projections are all different, since their kernels are the spans of $e_2+ce_1$ which are all different subspaces of $W$.
So, in your argument, while the projections $p_{g,\pi}$ and $\pi$ have the same range, this does not imply that they are equal, or that they have the same kernel.