The short answer is that you're right - the change in entropy due to the decrease in volume cancels out the change in entropy due to the increase in temperature. But you're also right that this involves a simultaneous change in $p$ and $T$, and it would be nice to know by how much each of these changes. Below I've done the calculation. It was a bit more involved than I was expecting, and I hope I didn't make any mistakes!
To do this calculation we first need to note that the ideal gas law,
$$pV=nRT,\tag{i}$$ tells us how pressure, volume and temperature relate to one another, but if you want to do any real calculations, you also need to know how the internal energy $U$ behaves. This is given by
$$
U =n\; c_V T,\tag{ii}
$$
where $c_V$ is the dimensionless heat capacity at constant volume. (Some people will define an ideal gas in such a way that $c_V$ is allowed to be a function of $U$, but here I'll assume it's constant. To a good approximation, $c_V=\frac{3}{2} R$ for a monatomic gas, or $\frac{5}{2} R$ for a diatomic one.) Equation $(\mathrm{ii})$ can't be derived from Equation $(\mathrm{i})$, so both of them are needed in order to define the properties of an ideal gas.
With this in mind, let's start with the fundamental equation of thermodynamics (for systems without chemical reactions):
$$
dU = TdS - pdV.
$$
Because we're considering an adiabatic process we know that $TdS = 0$, so
$$
dU = -pdV = - \frac{nRT}{V} dV,
$$
where the second equality is obtained by substituting the ideal gas law $(\mathrm{i})$. But we also know from the heat capacity equation $(\mathrm{ii})$ that $nT = U/c_V$. This gives us
$$
dU = - \frac{U\;R}{c_VV} dV,
$$
or
$$
c_V\frac{1}{U}dU = - R\frac{1}{V} dV.
$$
Now we can integrate both sides:
$$
c_V \int_{U_1}^{U_2} \frac{1}{U}dU = -R\int_{V_1}^{V_2} \frac{1}{V}dV,
$$
or
$$
c_V\left( \ln U_2 - \ln U_1 \right) =R \left( \ln V_1 - \ln V_2 \right).
$$
A quick note about interpretation is in order here. We're integrating both sides over different ranges ($U_1$ to $U_2$ and $V_1$ to $V_2$) and then setting them equal. This is because we want to know how much the internal energy will change if we reversibly change the volume by a certain amount, so we're looking for $U_2$ and $U_1$ as a function of $V_1$ and $V_2$. Anyway, now we can use some logarithm identities to get
$$
c_V \ln \frac{U_2}{U_1} = R\ln \frac{V_1}{V_2}
$$
or
$$
\ln \left(\frac{U_2}{U_1}\right)^{c_V} = \ln \left(\frac{V_1}{V_2}\right)^R,
$$
so
$$
\left(\frac{U_2}{U_1}\right)^{c_V} = \left(\frac{V_1}{V_2}\right)^{R},
$$
or
$$
V_1^{R} U_1^{c_V} = V_2 ^{R} U_2^{c_V}.
$$
This means that, for a reversible process, the quantity $VU^{c_V}$ must remain constant. Now, finally, we can substitute $U$ from Equation $(\mathrm{ii})$ to get
$$
V^R(c_VnT)^{c_V} = \text{constant for an adiabatic process.}
$$
There's a factor of $(c_Vn)^{c_V}$ that we can ignore by incorporating it into the constant on the right hand side, so
$$
V^RT^{c_V} = \text{constant for an adiabatic process,}
$$
or
$$
V_1^R T_1^{c_V} = V_2^R T_2^{c_V}.
$$
Given any values for $V_1$, $V_2$ and $T_1$, you can use this to work out $T_2$. By substituting into $(\mathrm{i})$ you can also work out the change in pressure.
A process is said to be reversible if the system can be brought back to its initial state without leaving any changes in the surroundings, i.e, the system and the surroundings can be brought back to their initial states. There are no conditions on it concerning the existence of heat exchange, so there is no problem in having an adiabatic or a non adiabatic reversible process (theoretically).
However, in practice, reversible processes themselves are impossible. Just as a frictionless surface in mechanics and a wire with zero resistance in electricity, a reversible process is an ideal case used to simplify problems, but doesn't exist in reality.
Best Answer
By definition a reversible adiabatic system has $dQ = 0$.
We also know the following from the Clausius Theorem :
$dS = \frac{dQ}{T}$
Then it is easy to see that there can be no change in entropy.
Note that irreversible adiabatic systems CAN see a change in entropy because in that case the above equation is no longer an equality but an inequality :
$dS < \frac{dQ}{T}$