Photons interact with matter if the matter offers quantum transitions that match, or nearly match, the photon's energy in the inertial frame of the matter. Ordinary matter such as wood, stone, etc. offers several groups of possible quantum transitions.
- Rotation of molecules (if they are free to rotate, i.e., not condensed matter)
- Vibration of molecules - bending, quivering actions
- Electronic excitations
- Nuclear excitations (there being various kinds, ignored here for simplicity)
Microwaves have such low energy they can't do much, though they might excite some types of vibrations on larger floppier molecules - however, any type of molecule that could be described as "floppy" probably isn't good for construction materials. Rotational modes aren't possible in a strong material made of crosslinked polymers or silicates. So microwaves mostly fly right through.
Near-infrared and visible light can kick electrons into higher molecular orbitals. Even if the energies aren't a match, just close, there is interaction, as Heisenberg lets them cheat temporarily. Also, having more energy, visible light photons can stir up a greater variety of vibrational modes. There's nothing in common wall materials to prevent that, and in fact, the interaction with photons is so strong that the material, if not super-thin (microns), will be opaque. Of course, glass is an exception.
Gamma rays are of such high frequency, electrons (or ions, or polarized ends of molecules) can't keep up due to inertia - so no interaction, or only a little. At the right frequencies, gamma photons can interact with nuclei, but for a randomly chosen source of gammas, its photons are unlikely to match closely enough with any of the available nuclear excitations, and can't really do much at the molecular level - therefore, the material is almost transparent.
All this is so oversimplified...
You are thinking in terms of atoms and molecules and you are mainly talking of solid state matter .
Solid state is another quantum mechanical phase, it has lattice structure with much smaller energies than atomic and molecular transition structures. Lattices have vibrational levels which are mainly responsible for the black body radiation solids emit, infrared is also photons.
A rule of thumb with radiation impinging on solids is that if the wavelength is smaller than the lattice dimensions the photons can penetrate easily the lattice, interacting only with direct scatters hence the higher penetration of X rays and gamma rays. Here is an article that discusses the penetration of radiation, X rays and higher.
For glass and optical frequencies there is a good answer here in this site., essentially the structure of the transparent materials is such that the photons pass through without loosing energy in the visible.
For infrared where the wavelengths are large in comparison with lattices or distances between molecules in liquids, the photon can give up its energy in collective excitations at the surface gradually heating up the material.
For ultraviolet, glass, depending on the type, has some absorptive bands, the photon energy transferred at the surface to collective modes or breaking molecular bonds and transformed to heat ( infrared) further in.
So your
Once you reach a critical frequency, however, the photons will begin to be absorbed because they have enough energy to excite the electrons (which is why glass is opaque in ultra-violet).
has small probability to happen until x-ray energies are reached which are the energies of bound electrons, and the link above gives the dependence in a simplified manner.
Best Answer
This isn't true. Gamma rays have energies of $\gtrsim 100$ keV, which is orders of magnitude higher than the energy required to ionize matter. They are ionizing radiation.
You may be misunderstanding something you've heard about the fact that some gamma rays are highly penetrating. Gamma rays interact with matter via the photoelectric effect, Compton scattering, and pair production. Each of these processes has some probability, which depends on the energy of the gamma, and also, for some of these processes, on the atomic number of the atom. Low-energy gammas, bordering on the x-ray region, are relatively non-penetrating, but higher-energy gammas will typically pass through a fairly large amount of matter without interacting. For example, one might use a lead brick as shielding against high-energy gammas. So in this sense, high-energy gammas may be relatively unlikely to produce ionization in a given object, if the object is not very thick and has a low atomic number.