Gauss' law is applicable for a finite wire. But, it's useless in this case.
In the infinite example, you assume some things due to symmetry, namely:
It's pretty obvious why these things can be assumed--moving up and down the wire should not change $\vec E$, so we take it constant. Also, there should be no direction bias, so $\vec E$ has no component along the wire.
In this way, $\oint \vec E.\mathrm dS\to\oint|\vec E|\mathrm dS_{curved}$ (since it's perpendicular), and then $\oint|\vec E|\mathrm dS_{curved}\to |\vec E| \oint\mathrm dS_{curved}$ (since it's constant at a given radius)
Once E is outside the integral, we can easily integrate the $\mathrm dS_{curved}$ term.
On the other hand, for a finite wire, we have:
The translational symmetry along the wire is lost (while the symmetry under rotations and the symmetry under reflection through the origin are preserved), and the form of $\vec E$ is not predictable{*}.
So we can't remove the dot product, and we can't take $\vec E$ out of the integral. Since the integral is a closed integral, it is sort of like a definite integral in that we can't just differentiate the equation to get an answer. So we can't solve the Gauss' law integral, so we're stumped. You have to use Coulomb's law and find it by taking elements.
*$\vec E=\frac{k\lambda}{R}\left((sin\alpha+sin\beta)\hat e_r +(cos\alpha-cos\beta)\hat k\right)$. See? Not predictable.
Both methods give the same answer, provided one applies them consistently. Coulomb's law actually reads
$$ \vec{E} = \frac{1}{4\pi\epsilon_0} \int_{-\infty}^\infty dx\,\frac{\lambda}{x^2 +r^2} \times \sin{\theta}\, \hat r,$$
where the factor of $\sin \theta$ is what picks out just the radial component, as required by symmetry. But since
$$\sin\theta = \frac{r}{\sqrt{x^2 + r^2}},$$
the integral we must evaluate actually has the form
$$ \frac{1}{4\pi\epsilon_0} \int_{-\infty}^\infty dx\,\frac{\lambda\, r}{(x^2 +r^2)^{3/2}} = \frac{1}{2\pi \epsilon_0} \frac{\lambda}{r}.$$
Comfortingly, this result agrees with the computation via Gauss's law.
Best Answer
You don't have to assume there is no axial component - it will become apparent when you do the derivation.
Let us assume, without loss of generality, that the line of charge extends in the $X$ direction. Now let us look at the electric field at a point $P$ due to a small line element $dx$, where there is a charge density $\rho$ per unit length. Without loss of generality we can put $P$ at the origin, and look at the wire which is displaced a distance $y$.
Now we can write the expression for the $E_x$ and $E_y$ fields at $P$ due to this element:
$$dE_x = \frac{1}{4\pi\epsilon_0} \frac{\rho dx}{r^2} \frac{x}{r}\\ dE_y = \frac{1}{4\pi\epsilon_0} \frac{\rho dx}{r^2} \frac{y}{r}$$
Writing $r=\sqrt{x^2+y^2}$ and integrating for a wire from $x=a$ to $x=b$ this becomes:
$$E_x = \int_a^b \frac{1}{4\pi\epsilon_0} \frac{\rho~ x~dx}{\left(x^2+y^2\right)^{3/2}}\\ E_y = \int_a^b \frac{1}{4\pi\epsilon_0} \frac{\rho~ y~dx}{\left(x^2+y^2\right)^{3/2}}$$
I will leave you to think about the details - but note that since the expression for $E_x$ is odd in $x$, any integral with symmetrical limits ($a=-b$) will be zero.
A more formal approach (formulated in a general case) can be found at this link. The integral shown there gives you the behavior in terms of the angles between the wire, and the lines connecting the ends of the wire to the point of interest; again, this shows the symmetric nature of the problem; and since these angles will tend to ± $\pi/2$ when the wire becomes infinitely long, the component along the wire will indeed disappear.