I'm interpreting your question as follows: when you write $H = \pi_1(Y,\ast)$, you mean the inclusion $i:Y\rightarrow X$ induces an injective map on $\pi_1$ with image $H$. (As opposed to $\pi_1(Y)$ being abstractly isomorphic to $H$).
With this interpretation, the answer is no. For example, consider $X = S^1$. It is well known that $\pi_1(X) = \mathbb{Z}$. Now, consider the subgroup $H = 2\mathbb{Z}$. I claim that there is no proper subspace $Y$ for which which has this fundamental group.
We may assume wlog that $Y$ is connected. Then note that any connected proper subset of $S^1$ is homeomorphic to a connected subset of $(0,1)$. These are easy to classify, they are, up to homeomorphism, $(0,1)$, $[0,1]$, and $(0,1]$. None of these has infinite cyclic fundamental group.
Edit Here is an example with $H$ not even abstractly isomorphic to a particular subgroup of $\pi_1(X)$.
Take $X = S^1 \vee S^1$, the wedge sum of 2 $S^1$s. The fundamental group of $X$ is known to be isomorphic to the free group on two generators. It's also know that a free group on two generators contains subgroups isomorphic to the free group on $n$ generators for any finite $n$ (we make even take $n$ to be countable). Further, it's know that free groups on different numbers of generators are never isomorphic.
Let $H$ denote any of these subgroups for $n > 2$. In particular, $H$ is not isomorphic to either $0$, $\mathbb{Z}$, or the free group on two generators. I claim that no subspace $Y$ of $X$ has $H$ as a fundamental group, even up to abstract isomorphism.
As above, we may assume $Y$ is connected. If $Y$ does not contain the wedge point, then it must be contained in a proper portion of one of the two circles, so the above argument shows $\pi_1(Y)$ is trivial. Hence, $Y$ must contain the wedge point. Now, if $Y$ does not contain the whole of the first circle, then $Y$ deformation retracts onto a subspace of the other circle, hence, by the previous argument has $\pi_1 = 0$ or $\mathbb{Z}$. Thus, $Y$ must contain the while first circle. Likewise, $Y$ must contain the whole second circle.
But then this implies $Y = X$, so $Y$s fundamental group is isomorphic to the free group on $2$ generators, so not isomorphic to $H$.
Final (?) Edit Here's one in the finite fundamental group case. Let $X$ be obtained from $S^1$ by attaching a $D^2$ by a degree $4$ map. A simple van Kampen argument shows $\pi_1(X) = \mathbb{Z}/4$. Let $H$ be the unique subgroup of $\pi_1(X)$ isomorphic to $\mathbb{Z}/2$.
I claim that no subspace $Y$ has fundamental group abstractly isomorphic to $H$. If $Y$ misses a point of the interior of the $D^2$, then $Y$ deformation retracts onto a subspace of $S^1$, so by the above argument, has $\pi_1 = 0$ or $\mathbb{Z}$. Hence, we may assume wlog that $Y$ contains all of the interior of $D^2$. Likewise, if $Y$ misses a point of $S^1$, then restricting the van Kampen argument to $Y$ shows that $\pi_1(Y) = 0$, so $Y$ must contain all of $S^1$. This implies $Y = X$, so $\pi_1(Y)\neq H$.
Your idea is correct, but the moving $p$ is detail to be taken care of. I like to fix bad homotopies by messing with the square, since the space is too complicated. Imagine starting at the bottom-right corner of the square. Then going backwards along the bottom is going along the path $\sigma(t)$ of the point $p$. That is, $\sigma(t) = h(t,p)$. If we then go up, this goes along our path $\gamma$, and finally we go along the top, tracing $\sigma$, in the forward direction. So your square seems to be a "weird homotopy" from $\sigma^{-1} \ast \gamma \ast \sigma$ to the constant path, with the former path along three of the sides instead of just one. However, I claim that a clever transformation $I^2 \rightarrow I^2$ can turn this "weird homotopy" you defined into an honest homotopy from $\sigma^{-1} \ast \gamma \ast \sigma$ to the constant path.
EDIT: You must, of course, then figure out how to move from this to the fact that $\gamma$ is homotopic to the constant path. For this, you will either apply a nice theorem or attempt another fudge map for your square.
Best Answer
Here's a general construction:
Claim: $\pi_1(X) \cong G$.
Now to answer your question, just take any finite $G$. For a nice example, work out the construction where $G$ has a single generator, i.e. is cyclic.
(It's worth pointing out that with a little more work you can prove that actually every finitely presented group is the fundamental group of some compact $4$-manifold.)
For examples of $X$ that are not contractible, note that if $\pi_1(X) \neq 0$ then certainly $X$ is not contractible, so my above construction provides examples. Moreover if $\pi_1(X) = 0$ but some higher homotopy group is non-zero then it will still be not contractible. For example if $n>1$ then $\pi_1(S^n) = 0$ so the exercise you did verifies that any continuous map $S^n \to S^1$ is nullhomotopic for $n>1$, or in other words $\pi_n (S^1) = 0$ in this range, but still $\pi_n(S^n) \cong \mathbb{Z}$ so it is not contractible. There are many other examples of spaces where $\pi_1(X) = 0$ but $X$ is not contractible, even spaces where all of the homotopy groups vanish (for example the Warsaw Circle).