To say the same thing David Zaslavsky said in slightly different words, the second law implies that entropy cannot be destroyed, but it doesn't prevent you from moving it around from place to place. When we write the equation $\Delta S = \int_a^b \frac{dQ}{T}$, we're assuming that this $dQ$ represents a flow of heat into or out of the system from somewhere else. Therefore $S$ (which, by convention, represents only the entropy of some particular system) can either increase or decrease. Since we're talking about a reversible process, the entropy of some other system must change by an equal and opposite amount, in order to keep the total constant. That is, $\Delta S + \Delta S_\text{surroundings} = 0$.
One other thing: in thermodynamics, "closed" and "isolated" mean different things. "Isolated" means neither heat nor matter can be exchanged with the environment, whereas "closed" means that matter cannot be exchanged, but heat can. In your question you say the second law "prohibits a decrease in the entropy of a closed system," but actually this only applies to isolated systems, not closed ones. When we apply the equations above, we're not talking about an isolated system, which is why its entropy is allowed to change. I mention this because you said you're teaching yourself, and in that case it will be important to make sure you don't get confused by subtleties of terminology.
The short answer is that you're right - the change in entropy due to the decrease in volume cancels out the change in entropy due to the increase in temperature. But you're also right that this involves a simultaneous change in $p$ and $T$, and it would be nice to know by how much each of these changes. Below I've done the calculation. It was a bit more involved than I was expecting, and I hope I didn't make any mistakes!
To do this calculation we first need to note that the ideal gas law,
$$pV=nRT,\tag{i}$$ tells us how pressure, volume and temperature relate to one another, but if you want to do any real calculations, you also need to know how the internal energy $U$ behaves. This is given by
$$
U =n\; c_V T,\tag{ii}
$$
where $c_V$ is the dimensionless heat capacity at constant volume. (Some people will define an ideal gas in such a way that $c_V$ is allowed to be a function of $U$, but here I'll assume it's constant. To a good approximation, $c_V=\frac{3}{2} R$ for a monatomic gas, or $\frac{5}{2} R$ for a diatomic one.) Equation $(\mathrm{ii})$ can't be derived from Equation $(\mathrm{i})$, so both of them are needed in order to define the properties of an ideal gas.
With this in mind, let's start with the fundamental equation of thermodynamics (for systems without chemical reactions):
$$
dU = TdS - pdV.
$$
Because we're considering an adiabatic process we know that $TdS = 0$, so
$$
dU = -pdV = - \frac{nRT}{V} dV,
$$
where the second equality is obtained by substituting the ideal gas law $(\mathrm{i})$. But we also know from the heat capacity equation $(\mathrm{ii})$ that $nT = U/c_V$. This gives us
$$
dU = - \frac{U\;R}{c_VV} dV,
$$
or
$$
c_V\frac{1}{U}dU = - R\frac{1}{V} dV.
$$
Now we can integrate both sides:
$$
c_V \int_{U_1}^{U_2} \frac{1}{U}dU = -R\int_{V_1}^{V_2} \frac{1}{V}dV,
$$
or
$$
c_V\left( \ln U_2 - \ln U_1 \right) =R \left( \ln V_1 - \ln V_2 \right).
$$
A quick note about interpretation is in order here. We're integrating both sides over different ranges ($U_1$ to $U_2$ and $V_1$ to $V_2$) and then setting them equal. This is because we want to know how much the internal energy will change if we reversibly change the volume by a certain amount, so we're looking for $U_2$ and $U_1$ as a function of $V_1$ and $V_2$. Anyway, now we can use some logarithm identities to get
$$
c_V \ln \frac{U_2}{U_1} = R\ln \frac{V_1}{V_2}
$$
or
$$
\ln \left(\frac{U_2}{U_1}\right)^{c_V} = \ln \left(\frac{V_1}{V_2}\right)^R,
$$
so
$$
\left(\frac{U_2}{U_1}\right)^{c_V} = \left(\frac{V_1}{V_2}\right)^{R},
$$
or
$$
V_1^{R} U_1^{c_V} = V_2 ^{R} U_2^{c_V}.
$$
This means that, for a reversible process, the quantity $VU^{c_V}$ must remain constant. Now, finally, we can substitute $U$ from Equation $(\mathrm{ii})$ to get
$$
V^R(c_VnT)^{c_V} = \text{constant for an adiabatic process.}
$$
There's a factor of $(c_Vn)^{c_V}$ that we can ignore by incorporating it into the constant on the right hand side, so
$$
V^RT^{c_V} = \text{constant for an adiabatic process,}
$$
or
$$
V_1^R T_1^{c_V} = V_2^R T_2^{c_V}.
$$
Given any values for $V_1$, $V_2$ and $T_1$, you can use this to work out $T_2$. By substituting into $(\mathrm{i})$ you can also work out the change in pressure.
Best Answer
The formula for the change in entropy $$ \mathrm{d}S = \frac{\delta Q_\mathrm{rev}}{T} $$ refers to the entropy change of the system. For a reversible process the total change in entropy of the system and its surroundings will be $0$. The reason for this is that for a reversible process temperature of the system and its surroundings must only differ infinitesimally (otherwise an infinitesimal change could not reverse the process) and clearly the heat transferred to the system is the negative of the heat transferred to the surroundings so \begin{align} \mathrm{d}S_\mathrm{sys} + \mathrm{d}S_\mathrm{sur} & = \frac{\delta Q_\mathrm{sys}}{T} + \frac{\delta Q_\mathrm{sur}}{T}\\ & = \frac{\delta Q_\mathrm{sys} - \delta Q_\mathrm{sys}}{T}\\ & = 0 \end{align}