Because what you are doing is a flow process, with mass inflow and no mass outflow, you need to use the thermodynamic equation:
$dU_{cv}={m_{in}d}{H}_{in}-{m_{out}d}H_{out}+\delta Q-\delta W_{shaft}$
If you insulate your air cylinder well enough, $\delta Q = 0$.
Assuming that your air cylinder does not deform, $\delta W = 0$.
Since you are filling your cylinder with air and assuming no air escapes, ${m_{out}d}H_{out} = 0$
Therefore, the enthalpy of the gas which you are filling adds to the internal energy of the gas in the cylinder, and because the internal energy is positively correlated to temperature, the temperature in of the gas in the cylinder rises.
$\Delta{U_{cv}}={H_{in}}>0$, so
$\Delta{T} >0$
You may apply the reverse for the release of air from the cylinder. In this case:
$\Delta{U_{cv}}={-H_{out}}<0$, so
$\Delta{T} <0$
http://en.wikipedia.org/wiki/Thermodynamic_system#Flow_process
Here is Feynman's intuitive explanation: rubber contains very long molecules like chains. nearby atoms continuously hit this chains. of course you can imagine the stronger hitting be, the shorter will be chain. now heating rubber makes atoms faster, make them hit stronger which makes chains and so rubber shorter.
Best Answer
This is a very interesting question with a very interesting answer. The key lies in the reason for the stretchiness of the rubber band.
Rubber is made of polymers (long chain molecules). When the elastic band is not stretched, these molecules are all tangled up with each other and have no particular direction to them, but when you stretch the elastic they all become lined up with one another, at least to some extent. The polymer molecules themselves are not stretched, they're just aligned differently. To a first approximation there's no difference in the energy of these two different ways of arranging the polymers, but there's a big difference in the entropy. This just means that there's a lot more different ways that the polymers can be arranged in a tangled up way than an aligned way. So when you release the elastic band, all the polymers are jiggling around at random due to thermal motion, and they tend to lose their alignment, so they go back towards the tangled state, and that's what makes the elastic contract. This is called an entropic force.
Now, I said earlier that there isn't any difference in energy between the stretched (aligned) and un-stretched (tangled) states. But it takes energy to stretch the elastic -- you're doing work to pull the ends apart, against the entropic force that's trying to pull them back together. That energy doesn't go into stretching the individual polymer molecules, but it has to go somewhere, so it ends up as heat. Some of this heat will stay in the elastic (making the polymer molecules jiggle around a bit faster) but some will be transferred to the surrounding air, or to your skin.
The reverse happens when you let the elastic contract. The molecules are jiggling around at random and becoming more and more tangled, which makes them contract. But to contract they have to do work on whatever's holding the ends of the elastic apart. That energy has to come from somewhere, so it comes from heat.
At first this might seem to run against thermodynamics - normally you can't just cool something down without heating something else up. But remember that the state with the tangled molecules has a higher entropy than when they're aligned. So you're taking heat out of the air, which reduces its entropy, but this reduction in entropy is countered by the increase in entropy of the elastic itself, so the second law is safe.
For further reading you can look into the ideal chain, which is an idealised mathematical model of this situation.