The probability for a photon of frequency $\nu$ to be absorbed by an atom and excite (or ionize) it to a given state is given by the cross section $\phi(\nu)$ of that particular transition. This cross section is not a traditional geometric cross section, but it loosely corresponds to this, and has units of area.
If $\phi(\nu)$ were a delta-function — i.e. if it had a non-zero value only at an exact frequency — a photon off resonanance would indeed pass right through the atom without interacting. However, due to the finite lifetime $\Delta t$ of the excited state, the energy of that state is not exact, but has an uncertainty $\Delta E$ associated with it. This uncertainty satisifes the Uncertainty Principle $\Delta E \Delta t \gtrsim \hbar/2$.
Hence, there is also an uncertainty $\Delta \nu = \Delta E / h$ in the frequency needed to excite it, and this uncertainty will broaden the line profile to a very narrow, but finite, width. That means that the probability of off-center photons to excite the state quckly approaches zero, the farther from the line center the photon is, but in principle there will always be a non-zero probability.
Natural broadening
The line profile thus has a so-called natural broadening, given by a Lorentzian profile
$$
\phi(\nu) \propto \frac{1}{(\nu-\nu_0)^2 + a^2},
$$
where $\nu_0$ is the central frequency, and $a$ is the broadening, or damping, parameter.
Thermal broadening
In general, an ensemble of atom will move with random velocities due to their temperature. Their velocity distribution along a line of sight is given by a Gaussian, i.e. they're normally distributed. This means that if a photon entering the ensemble has a frequency that is, say, too high for a significant probability of absorption, then there will probably be some of the atoms that happen to move with a velocity in the same direction as the photon, such that, in the reference frame of the atom, the photon is redshifted into resonance, and is absorbed anyway.
Hence, the effective cross section of the average atom is not a Lorentzian, but a convolution of a Lorentzian and a Gaussian profile — a so-called Voigt profile. This profile is dominated by the thermal motion in the line center, and by the natural broadening in the wings.
The profiles are shown below, normalized to unity in the line center. The value on the $x$ axis is basically frequency offset from the line center. Note the logarithmic axis; on a linear scale it would look much more peaked (Laursen 2010).
Resonance scattering
My favorite example is the Lyman $\alpha$ photon, the which energy of which corresponds the the energy difference between the ground state and the first excited state of the hydrogen atom. A Ly$\alpha$ photon incident on a neutral hydrogen atom will excite it, and after $\sim10^{-8}\,\mathrm{s}$ the electron de-excites and emits another Ly$\alpha$ photon in some direction. The scattering is coherent in the reference frame of the atom; i.e. it leaves the atom with the same energy as it entered. However, in the reference frame of an external observer, if a high-refquency photon is scattered by a fast-moving atom, then if by chance it is scattered back in the direction from which it came, the motion of the atom will now, in the external frame, redshift it. Thus, Ly$\alpha$ photon scattering their way through a cloud of neutral hydrogen in a galaxy will also diffuse slowly to higher and lower frequencies, and when they escape the galaxy, the spectrum will tend to have been split up in a double-peaked profile, such as this one (Verhamme 2008):
Phase function
The direction into which the photon is scattered does to some extend depend on the exact frequency of the photon. The probability distribution for the direction is called the phase function. For Ly$\alpha$ photon, photons in the line center can be excited to two different states; one state results in completely random scattering, while the other preferentially scatter wither forward or backward. For photons in the wings, the scattering is always governed by an anisotropic phase function.
Best Answer
There is an important concept of the oscillator called resonance. For example you can place an energized "c" note tuning fork on the "c" piano string .... and the string will oscillate .... However if you place the fork on the C sharp or B flat string these strings will NOT oscillate.
The photon has energy-frequency and it is pure vibration (wave) in the EM field ... when a suitable electron can resonant at the frequency it can absorb the pure energy. When the photon is at a different frequency-energy the suitable electrons just don't resonate or absorb the energy.
All of quantum mechanics is based on the oscillator model for the atom/electron orbital behaviour.