ANY point charge in this situation DOES contribute to the field inside the cavity. Negative charges on the interior wall don't contribute to the flux through a surface inside the cavity, which is not to be confused.
Here are more precisions :
1) The misconception you seem to have is a classical one : you look at an equation stating that $A = B$ and you tend to interpret it as "B causes A" when in fact it only says "B equals A". In this case, the flux (surface integral of the E-field) is EQUAL to the interior charges on epsilon does NOT means that the E-field is only caused by the interior charges. In fact, ALL charges contribute to the E-field even though its flux is equal to a multiple of the interior charges.
Here's a counter example that makes this point obvious : consider a long charged wire with charge density $A$. To find the E-field, one would use a cylindrical gaussian surface of length $L$ and assume that the total interior charges are $AL$. Then one would assume that the E-field has the same cylindrical symmetry has the wire, which would simplify the surface integal to E multiplied by the lateral surface $2\pi r L$. Now, part of the wire is obviously outside the gaussian surface. What if we remove all the charges there ? The interior charges would remain the same, but the symmetry would be lost, making it impossible to reduce the integral to $E 2\pi r L$. In fact, part of the flux would now be through the ends of the cylindrical surface. That shows the the "E" in the integral is the resulting E caused by ALL charges.
2) in the specific case of a spherical uniform distribution of charges, with or without a cavity, isolating or conducting, the resulting E-field cancels inside the cavity. This can be shown without Gauss law, using superposition. It was done by Newton for the first time (he considered gravity, but the maths is the same : consider any point inside the cavity and diametrically opposed solid angles radiating from that point. These angles enclose a small patch of charges. the size of the patch is proportional to $r^2$ but the E-field that the patch generates at the considered point goes as $\frac{1}{r^2}$. So the E-field produces by the patch doesn't depend on $r$. That means that patches in opposing directions ALWAYS cause cancelling E-field, wherever the point you're considering is located inside the cavity).
tl;dr: thats not how conductors behave.
I am guessing that your doubt stems from the following notion: oppositely charged surfaces when connected with a conductor equilibrate.
So the induced charges, being connected by a conductor's bulk, should merge and vanish!
After all, initially, when there wasn't any charge inside the shell, the shell itself was uncharged. So where were these charges then? They must have been in happy pairs.
So how come the presence of a charge inside the shell ripped them apart? It must be that the force experienced by the +ve and -ve charges being opposite, rips them apart, right? The negative ones move to the inner surface, the positive ones to the outer.
But this can't be true as charges which have initially paired would be so strongly held that no external field, let alone of some measely $Q$, would pull them apart.
Also, how many of these pairs of charges are their anyways? Can we support $10Q$ inside? What about $10^{10}Q$? In theory, $\pm10^{10}Q$ would just pop out of the bulk of the conductor, ready for their surface duty, right?
But surely this can't be true, if for example, we took more charge than their was in the conductor to begin with.
You see, we run into all kinds of trouble assuming that a conductor is a free see/source of as many induced* charges as you want with opposite charges also being induced automatically.
So what now?
An ideal argument in electrostatics should be independent of phenomenology but I can't seem to find a simpler way to clarify you query. The crux is that conductors are made of neutral atoms with delocalized electrons. By delocalized we mean free to move within the bulk of the conductor.
So when you put $+Q$ inside, the free $e^-$s just gather as close as they can to it--on the inner surface. Thats the only imperative they have--as opposite charges attract. If you had put $-Q$, they would have moved away, as far as they could, to the outer surface, (or even to $\infty$ if you grounded the outer surface). Note that there isn't any converse movement of positive charges. Since they already were free, there aren't any pairs to rip them off of. So what about the induced positive charges? They are the exact** locations the $e^-$s left--remember the atoms are neutral.
So you see there isn't any reason the electrons should feel the urge to go back to the outer surface as long as they are pinned to the inner one.
So what about the case when two oppositely charged surfaces are connected by a conductor? How is that different? Oh no..there, $e^-$s have been put on the $-ve$ conductor and removed from the +ve one. So upon connecting, them electrons feel attracted to the +ve atoms and go to them.
*Note that induced $\ne$ created $\because$ charge conservation
**upto lattice spacing
Best Answer
Ok i think i found the answer using image charges.
For finding the potential, we can do it by using an image charge $q'$ placed appropriately, such that the potential from $Q$ and $q'$ is zero everywhere on the surface of the conducting spherical cavity.
From a well known example, if we have a conducting sphere of radius R and a point charge with distance d from the center, then we can achieve zero potential on the surface of the sphere by placing an image charge $Q'=-QR/d$ at a distance $d'= R^2/d$. So if we invert the charges and their positions we have what we're looking for:
We place an image charge $q'=-Qa/R$ at distance $d=R^2/a$ from the center of the caavity. Now the potential on the surface is zero, as is the real potential everywhere outside the sphere. Now we can use the formula $$V(x,y,z)={1\over 4 \pi \epsilon_0 }({q\over r_q}+{q'\over r_q'})$$ to find the potential function inside the cavity, taking (0,0,0) to be the canter: $$ V(x,y,z)={1\over 4 \pi \epsilon_0 }({q\over \sqrt{(x-a)^2+y^2+z^2}}-{qR\over a \sqrt{(x-d)^2+y^2+z^2}})$$ for $\sqrt{x^2+y^2+z^2} \le R$
After that we easily find the field from $E=-\nabla V$