I think there's something wrong with your statement that $(X,A)$ has the homotopy extension property if $X$ is Hausdorff and $A$ is closed. Take the Hawaiian earring space $E$ and form a new space $X$ by joining two copies of $E$ by an edge from the wedge point to the wedge point. Then contracting the central edge to get a new space $E\vee E$ is not a homotopy equivalence. For example, it is not surjective on the fundamental group. There are loops in $E\vee E$ which travel back and forth over each copy of $E$ infinitely many times. However there are no such paths in $X$ since they would have to travel over the central edge infinitely many times.
Hatcher assumes $(X,A)$ are a CW pair, which is a far stronger condition.
I think that you are correct in the hint that you're remembering that the exercise told you to consider.
My idea is that, using this hint, you ought to be able to proceed inductively in constructing the regular complex $Y$.
Start by looking at all attaching maps $\varphi_{\alpha}:S^{\alpha} \rightarrow X_0$ onto the $0$-skeleton for $X$. Since $X$ is finite, these can be listed: $\{\varphi_{\alpha_1},...,\varphi_{\alpha_r}\}$.
Now, as you suggest, for each of these we wish to somehow consider instead the map $S^{\alpha} \rightarrow X_0 \times D^{\alpha+1}$, because this would be injective. What we therefore do is, for each $\alpha_1,...,\alpha_r$, take the successive product of $X_0$ with each of the $D^{{\alpha_i}+1}$s. This results in a space
$$Y_0 := X_0 \times D^{{\alpha_1}+1} \times....\times D^{{\alpha_r}+1}.$$
We can now use the standard cellular decomposition of $D^{k+1}$ to obtain a cellular decomposition of the product above. This is the "bottom" of our new regular complex $Y$. It is not the $0$-skeleton for our CW-complex $Y$, but it is homotopy equivalent to the $0$-skeleton of $X$. As such, we do indeed mildly abuse notation by calling this thing $Y_0$.
We then proceed with attaching cells to this product via the maps $\psi_{\alpha}$ as you suggest above. Importantly, we proceed as we normally would when we construct a CW-complex. We first attach the $1$-cells, and procede with $2$-cells, etc.
It is important to note a couple of things at this point.
We have certainly dealt with all attaching maps
for the $1$-skeleton of $X$ at this point. What this means is that
this process can indeed now be iterated; we look at attaching maps onto $X_1$, take whatever products we need to take with disks D^{\alpha}$ in order that these maps will be injective, and continue.
Importantly what we have constructed (and what we construct inductively in
subsequent steps) is homotopy equivalent to $X_0$, $X_1$ and so on.
It is indeed easy enough to see that the product $X_0 \times
D^{{\alpha_1}+1} \times....\times D^{{\alpha_r}+1}$ is homotopy
equivalent to $X_0$, but what about the next steps? We wish to see that $Y_1$ is homotopy equivalent to $X_1$. This can be seen in a few different ways, but the simplest for these purposes is just to state the following lemma (which I state rather wordily):
Lemma: Due to the nature of the construction of CW-complexes,
whatever space $X$ you're attaching a particular cell to can be replaced by a
homotopy equivalent space $\overline{X}$ (and the attaching map by the appropriate homotopic map) and the space which results $\overline{X} \cup D^{k}$ is homotopy equivalent to $X \cup D^k$.
Now what we're essentially doing is replacing $X_0$ with the homotopy equivalent $Y_0$, and attaching each $1$-cell to this new thing. Utilising the above lemma for each cell we attach gives that the resulting space is homotopy equivalent to $X_1$. We now "fatten up" this resulting space in exactly the same manner as how we "fattened up" $X_0$, by taking products with disks. Taking products with disks trivially results in a homotopy equivalent space, and what we end up with is the space which we will label $Y_1$; homotopy equivalent to $X_1$.
I think that iteratively doing this should complete the proof.
I should note that whilst the $Y_0$ and $Y_1$ which I mention above are certainly not the $0$ or $1$ skeletons for the space $Y$, there is nevertheless a natural $CW$-complex structure available for $Y$ which is obtained by using the standard cellular decomposition of the disks $D^k$, and this is then indeed a regular complex.
Best Answer
Let $X$ be a CW-complex and $A$ a subcomplex, that means $A$ is a union of cells $\{e_\alpha^n\mid\alpha\in J \}$ such that $\overline{e_\alpha^n}$ is also contained in $A$. Then $A$ is closed and the quotient map $q:X\to X/A$ is a closed surjection. Such maps preserve the $T_4$-property of $X$. In particular, $X/A$ is Hausdorff. We can then apply the following proposition which translates between the intrinsic and the constructive definition of a CW complex:
As the new characteristic maps we take $\Psi_\beta=q\circ\Phi_\beta$ for each cell $e_\beta^n$ which is not in $A$, and a single map $D^0\to\{A\}$ as the new $0$-cell the set $A$ was shrunk to. These maps clearly satisfy condition (i). It is also easy to see that they satisfy (ii) since the image of $\partial D_\alpha^n$ can only be in fewer cells after collapsing $A$. Condition (iii) says that $X$ has the final topology of all $\Phi_\alpha$, but since $q$ is a quotient map, also $X/A$ will have the final topology of all $\Psi_\beta$.
So this gives us a natural cell complex structure on the quotient space.