Everything you say is correct: the sense that a finite group is "built" from its simple Jordan-Hölder factors is by repeated extensions. But this "building" process is much more complicated for groups than the analogous process of building integers from prime numbers because given a (multi-)set of building blocks -- i.e., a finite list $\mathcal{H} = \{ \{H_1,\ldots,H_n\} \}$ of finite simple groups -- there will be in general several (finitely many, obviously, but perhaps a large number) nonisomorphic groups $G$ with
composition factors $\mathcal{H}$. The simplest example of this has already been given by Zhen Lin in a comment: if
$\mathcal{H} = \{ \{ C_2, C_2 \} \}$,
then the two groups with these composition factors are $C_4$ and $C_2 \times C_2$.
It seems to be a working assumption of experts in the field that it is hopeless to expect a nice solution to the extension problem. For instance, consider the special case $\mathcal{H} = \{ \{ C_p,\ldots,C_p \} \}$, in which every composition factor is cyclic of order $p$ -- i.e. a finite $p$-group. It is known that the function $f(p,n)$ which counts the number of isomorphism classes of finite groups of order $p^n$ grows very rapidly as a function of $n$ for any fixed $p$. For instance, see here for a reference to the fact that $f(2,9) = 10494213$.
Nevertheless the group extension problem is an important and interesting one -- it is one of the historical sources for the field of group cohomology and still plays a major role -- and in many special cases one can say something nice. But the general "program" of classifying all finite groups by (i) classifying all simple groups and (ii) determining all finite groups with a given set $\mathcal{H}$ of composition factors does not seem realistic: step (i) was amazingly hard but in the end doable. It looks very easy compared to step (ii)!
Finally, you ask about infinite groups. Here the Jordan-Hölder theory extends precisely to groups $G$ which admit at least one composition series, and a standard (necessary and sufficient) criterion for this is that there are no infinite sequences of subgroups
$H_1 \subsetneq H_2 \subsetneq \ldots$
with each $H_i$ normal in $H_{i+1}$
or
$H_1 \supsetneq H_2 \supsetneq \ldots$
with each $H_{i+1}$ normal in $H_i$.
So for instance an infinite cyclic group $\mathbb{Z}$ does not satisfy the descending chain condition on subgroups and there is no sense (known to me, at least) in which $\mathbb{Z}$ is built up out of simple groups.
Best Answer
If the Galois group $ G $ of a finite Galois extension $ L/K $ where $ K $ has characteristic $ 0 $ is solvable, then letting $ n = [L:K] $, we may adjoin a primitive $ n $th root of unity $ \zeta_n $ to $ K $ to get the extension $ L(\zeta_n)/K(\zeta_n) $. This extension is also Galois, and its Galois group $ G' $ embeds into $ G $ by restricting automorphisms to $ L $. Since subgroups of solvable groups are solvable, it follows that $ G' $ is also solvable. Now, by first refining the composition series so that each successive quotient is cyclic, and then converting this into a chain of subfields by Galois correspondence, we get a chain
$$ K(\zeta_n) = L_0 < L_1 < L_2 \ldots < L_k = L(\zeta_n) $$
where each extension $ L_{i+1}/L_i $ is Galois with cyclic Galois group, and degree dividing $ n $. By Kummer theory, such extensions are obtained by adjoining the $ n $th root of some element in the ground field (this is why we adjoined $ \zeta_n $ before going through with this argument), thus each extension $ L_{i+1}/L_i $ is a simple radical extension, and the extension $ L(\zeta_n)/K(\zeta_n) $ is a radical extension. Since $ K(\zeta_n)/K $ is also a radical extension, it follows that $ L $ lies in some radical extension of $ K $. The converse direction is similar.