Why? Simple answer: because the electrostatic electric field owing to a point charge fulfils an inverse square law, or, equivalently, the electric potential $\phi$ from a point charge varies as $r^{-1}$.
If the potential variation were some function other than $1/r$, the statement wouldn't be true. See for example my answer here, where I discuss what would happen with other variations.
Because the potential owing to a point charge is $\phi\propto 1/r$ and the potential owing to a system of point charges is the superposition of their potentials, the potential $\phi$ fulfils the Laplace equation $\nabla^2\phi = 0$ at all points away from point charges and where the charge density is nought. Therefore:
$$\oint_{\partial V} \vec{E}\cdot \hat{n} \,{\rm d}S = -\oint_{\partial V} \nabla \phi \cdot \hat{n} \,{\rm d}S = -\int_V \nabla^2 \phi \,{\rm d}V = 0$$
by the divergence theorem, for the boundary $\partial V$ of any volume $V$ not containing charges.
Note that this would not work if the functional dependence of the potential owing to a point charge were different from $1/r$ because Laplace's equation would not be fulfilled.
With your statement: I am assumed you are asking "why do we have to worry about charge outside the surface if it doesn't add to the flux?" The answer is: because we may only be given the electric field on the surface: we may be given no other data. Therefore we can simply go ahead and calculate the flux through the surface to find out how much charge lies within: we don't want to have to be separating the electric field into that from charges inside the volume and that from charges outside. The whole point of such a calculation is to discover how much charge lies inside the volume.
In your last question it is important as to what you mean by
WRT the question.
If you are trying to find the E-field due to a point charge using Gauss then to make the surface integration easier you choose a surface which has the following properties:
In fact this is an equipotential surface.
Now if you move the Charge or the Gaussian surface you can still use Gauss but the surface integral becomes more difficult.
As the diagram shows the two conditions for a simple integration are not met in that the E-field is not perpendicular to the surface and it is not constant in magnitude.
In terms of flux it is perhaps easier to understand by going back in time when Faraday and others thought that the E-field lines (lines of force) actually "existed" and flux was a measure of the total number of these lines going through a surface. If you use that representation you can see that the number of field lines passing through the surface is the same in both cases, so the flux is the same.
So going back to your last question:
Also, would it matter (WRT the question) if the surface were a charged
conductor, or simply imaginary?
In the first diagram if you introduced a spherical conductor where the Gaussian surface is shown (remember that it is an equipotential surface) the E-field would not change except become zero inside the conductor.
However arbitrarily adding a spherical conducting shell will change the E-field
Charge $-q$ is induced on the inside of the shell with a non-uniform distribution but the amazing thing (to me) is that the charge $+q$ which is induced on the outer surface is uniformly distributed on the surface of the sphere. Perhaps not so amazing if one thinks that those charges are trying to get as far away from one another as possible.
So perhaps it is safer to keep the Gaussian surface as a figment of your imagination?
Best Answer
Gauss's law in general relates the total electric field flux through a closed surface to the charge enclosed in that surface. It does not give you the field at a certain point in space in general. You can, however, exploit symmetries in certain geometries (points, spheres, lines, cylinders, planes, etc.) to determine the field at certain points in space. If you can't exploit some symmetry to pull E out of your flux integral, then you can't use Gauss's law to determine the field at a point in space.
It's like saying "I have 10 numbers that add to 100." If this is all you know then you can't tell me which numbers I have used to get a total of 100. But if I also tell you "each number is the same number", then you can for sure say I used 10 10's to get a total of 100.
As for your diagram, yes the charge Q will influence the field at P. It will not change the total flux through your Gaussian sphere, or any other Gaussian surface not enclosing Q. You will not be able to use Gauss's law now to find the field at P since the symmetry that lets you pull E out of the integral is no longer present. (Although, you could argue that you can use Gauss's law on each charge individually and then use superposition to get the total field, but you can't use Gauss's law for the entire configuration to get the field at P in this case).