A simple solvable group must be cyclic of prime order (since it must be abelian, and so cannot have proper [normal] subgroups). But a simple solvable group would not contain nontrivial normal subgroups, so the proposition would be true for such a group by vacuity (the hypotheses are never satisfied). Alternatively, if you allow the whole group to be a "nontrivial normal subgroup", your $H$ can only be $G$ itself, which is already abelian, so you can set $A=H=G$; either way, the proposition is true for such a group.
(If $G$ is solvable, then $[G,G]$ is a proper subgroup of $G$; since it is always normal in $G$, if $G$ is also simple, then we must have $[G,G]=\{1\}$, hence $G$ is abelian).
Hint for the question. If $H$ is abelian, you are done. If not, then $[H,H]$ is nontrivial; use the fact that $H^{(n)}\subseteq G^{(n)}$, where $G^{(k)}$ is the $k$th term of the derived series of $G$, to show that $H$ is solvable, and use the fact that $H\triangleleft G$ to show $[H,H]\triangleleft G$. Then replace $H$ with $[H,H]$, lather, rinse, and repeat.
Different hint. Let $1=G_0\triangleleft G_1\triangleleft\cdots\triangleleft G_s = G$, with $G_i/G_{i+1}$ abelian. By Problem 8, you can pick the $G_i$ normal in $G$. Look at $H_i=H\cap G_i$.
Added. Sigh. I didn't notice that Problem 8 assumes $G$ is finite; the result is true, as witnessed by the derived series, but again it's probably not what you want.
Added 2. Okay, this should work; it takes some of the ideas of the hint in Problem 8 of the same page, so it should be "reasonable". Let $i$ be the largest index such that $G_i\cap H$ is a proper subgroup of $H$. Then $H\subseteq G_{i+1}$, hence $G_i\cap H\triangleleft H$. Moreover, $H/(H\cap G_i) \cong (HG_i)/G_i \leq G_{i+1}/G_i$, so $H/(H\cap G_i)$ is abelian. As in problem 8, this means that $x^{-1}y^{-1}xy\in H\cap G_i$ for all $x,y\in H$. Show that this is also true for all $G$-conjugates of $H\cap G_i$, hence their intersection, which is normal in $G$, contains all $x^{-1}y^{-1}xy$ with $x,y\in H$. If this is trivial, then $H$ is abelian and we are done. If it is not trivial, then this intersection is nontrivial, and normal in $G$. Replace $H$ with this intersection, and note that the largest index $j$ such that the intersection with $G_j$ is proper is striclty smaller than $i$; so you can set up a descent. Lather, rinse, and repeat.
EDIT: A user below points out a silly mistake I made. Make note of the changes.
This is the old trick that I mentioned here.
You get that $G$ acts on $P$ (where $P$ is your Sylow $3$-subgroup) by conjugation giving you a group map $G\to\text{Aut}(P)$ with kernel $C_G(P)$. Now, you know that $P$ is either $\mathbb{Z}_{9}$ or $\mathbb{Z}_3^2$. In the former case you have that $|\text{Aut}(P)|=|(\mathbb{Z}_9)^\times|=\varphi(9)=6$ and in the latter case you have that $|\text{Aut}(P)|=(3^2-1)(3^2-3)=48$ (this was computed because $\text{Aut}(\mathbb{Z}_3^2)\cong\text{GL}_2(\mathbb{F}_3)$ which has order $48$ by counting the number of vectors each column could be--they have to be lin. ind.). Either way, let's prove that $G/C_G(P)=\{e\}$.
The second case is easier. Namely, $|G/C_G(P)|$ divides both $|G|=315$ and $|\text{Aut}(P)|=48$. But, notice that since $P\leqslant C_G(P)$ one has that $9\mid C_G(P)$ and so $|G/C_G(P)|\mid 5\cdot 7$. So then, $|G/C_G(P)|\mid (5\cdot 7,48)=1$.
In the former case, you have that $|G/C_G(P)|$ divides both $|\text{Aut}(\mathbb{Z}_9)|=6$ and $G/C_G(P)$. But, you know that $P\subseteq C_G(P)$ so that $|G/C_G(P)|\mid 5\cdot 7$. So, you see that $|G/C_G(P)|$ divides both $35$ and $6$. Thus, once again, we see that $|G/C_G(P)|=1$ and so $G=C_G(P)$.
Either way we see that $G=C_G(P)$ and so $P\leqslant Z(G)$. Once again, we make the observation that $G/P$ is order $35$ which is cyclic. And, using the fact that if a quotient of $G$ by a subgroup of its center is cyclic, then $G$ is abelian, we may conclude that $G$ is abelian.
Best Answer
Note that $[x]$ is the greatest integer function. So $[x] = 0$ for all $x \in G$ as in question. Note that $G$ was the interval $[0, 1)$.