The first thing I would do is to reduce to abelian $p$-groups: If $A = G_1 \times \cdots \times G_r$ where each $G_i$ is a $p_i$-Sylow subgroup of $A$, then every subgroup $B$ of $A$ is a product of subgroups $B_i$ of $G_i$'s. (in fact $B_i$ will be the $p_i$-Sylow subgroup of $B$.)
So assume $A$ is an abelian $p$-group, say $A=\prod_{i=1}^k (\mathbb{Z}/p^{e_i} \mathbb{Z})^{r_i}$. In this case, clearly every $i$ can contributes up to $r_i$ direct factors of order dividing $p^{e_i}$. I think it is straightforward, although tedious, to show that no other subgroups can occur. (I haven't wrote the details, so one have to check...)
So I think the answer to your question is: YES
This has been, in some sense, answered in this recent MO question: https://mathoverflow.net/q/366008
The short answer is that if you consider exponent $p^n$ for $n\geq 7$, there are wildly many indecomposable pairs. There are infinitely many for $n=6$, according to Jeremy Rickard in that answer.
I wondered why I was having so much trouble even with the case $H$ cyclic! I do have a partial result in that case if it's of interest, but it gets very messy as the number of summands of $G$ increases. Everything is nice for two summands, becomes complicated for three summands, and then I can only write down heuristics after that. For three summands you need $n\geq 6$, which chimes with Jeremy's statement.
Edit: I will write down what I found for the cyclic case, although it's not so obvious now it's of any value.
Notice that being decomposable in the sense of the question is, for cyclic $H$, the same as being contained in an overgroup that has a complement. Call such an element a co-element (complemented overgroup), and otherwise an nco-element. If $x$ is a generator for $H$, it means that $x$ can be written as $(x_1,\dots,x_n)$, where $x_i$ are coefficients in a basis for $G$, and there exists a basis such that some $x_i$ is $0$.
So we can take the projection of $x$ onto some summand of $G$ and work with that. This immediately shows the following:
If $G_1$ has no nco-elements, and $G_1$ is a summand of $G$, then $G$ has no nco-elements.
From now on we will hunt for nco-elements.
We next see that $G=C_{p^n}\times C_{p^n}$ has no nco-elements. This follows easily: every element of $G$ has a root of order $p^n$, and elements of maximal order in abelian groups are complemented. (This is the start of a standard proof of the cyclic decomposition theorem for abelian groups.)
Thus $G=C_{p^{a_1}}\times \cdots \times C_{p^{a_r}}$, and all $a_r$ are distinct. Arrange so that $a_i>a_{i+1}$.
If $G=C_{p^a}\times C_{p^b}$ ($a>b$) and $x$ has no root of order greater than $p^b$ then $x$ is a co-element. Certainly we can take a root of $x$ of order $p^b$, so assume that $o(x)=p^b$. Note that $x=(x_1^{p^{a-b}\alpha},x_2^\beta)$. Such an element generates a complement to a subgroup of order $p^a$ unless $\beta$ is a multiple of $p$. But then $x$ has a $p$th root, a contradiction.
So $x$ can be chosen to have order greater than $p^b$, and cannot have order $p^a$ (as maximal-order elements are co-elements). Thus $a\geq b+2$.
Now there are lots of such elements. We just need to choose an element of order greater than $p^b$ that does not have a root of order $p^a$. For example, $(x_1,px_2)$ will do.
This solves the case of two summands for $G$. In general, we can choose any pair inside a decomposition of $G$ into summands and such an element must be an nco-element, otherwise we can complement it in that summand, and yield a complement in the whole group.
(This next bit is corrected, as noted by Jeremy in the comments.)
So the smallest possible case allowed by this is $C_{p^5}\times C_{p^3}\times C_{p}$, generated by $a,b,c$, and where $x$ should be something like $p^2a+pb+c$. That works for each of $\langle a,c\rangle$, $\langle b,c\rangle$ and $\langle a,b\rangle$. This gives all such elements for exponent $p^5$.
This yields:
Suppose that $G$ has an nco-element, and that $G$ is $d$-generator for $d\geq 3$. Then $G$ has exponent at least $p^5$. If $G$ has exponent $p^5$ then $d=3$ and the examples are exactly as above.
To generalize this you need a step of at least two in between the orders of the summands, and an increasing power of $p$ in the representation of the element $x$.
Best Answer
For a more general result:
Proof. "$\Rightarrow$" Let $K$ be a subgroup of $G\times H$. Let $e_G, e_H$ be neutral elements of $G, H$ respectively. Now let $(x,y)\in K$. Since $\gcd(|G|, |H|)=1$ then it follows that the order of $y$ and $y^{|G|}$ are the same. Thus $y^{|G|}$ generates $\langle y\rangle$. In particular we have
$$(x,y)\in K\ \Rightarrow$$ $$(x^{|G|}, y^{|G|})=(e_G, y^{|G|})\in K\ \Rightarrow$$ $$(e_G, y)\in K$$
The last implication because $y$ can be generated from $y^{|G|}$. Analogously if $(x,y)\in K$ then $(x, e_Y)\in K$.
Now if you consider projections
$$\pi_G:G\times H\to G$$ $$\pi_H:G\times H\to H$$
then you (obviously) always have
$$K\subseteq \pi_G(K)\times\pi_H(K)$$ What we've shown is the oppositie inclusion which completes the proof.
"$\Leftarrow$". Assume that $d=\gcd(|G|, |H|)\neq 1$. Let $p|d$ be a prime divisor and pick elements $x\in G, y\in H$ such that $|x|=|y|=p$ (they exist by the Cauchy's theorem). Now consider a subgroup $K$ of $G\times H$ generated by $(x, y)$. This subgroup is of prime order $p$. In particular if $K=G'\times H'$ then either $G'$ or $H'$ has to be trivial (because the order of product is equal to product of orders and $p$ is prime). But that is impossible since $(x,y)\in K$ and none of $x,y$ is trivial. Contradiction. $\Box$
So what it means is that if the orders of $G$ and $H$ are not relatively prime then there is a subgroup of $G\times H$ that is not a product of subgroups of $G$ and $H$. This should give you plenty of examples, e.g.
$$G=H=\mathbb{Z}_{n}$$ $$K=\langle(1,1)\rangle$$