This is because of Kirchhoff's law of thermal radiation. The corollary from it is that emissivity of a material is equal to its absorptivity.
As diamond is transparent even at large temperatures, which can be seen in this answer, its absorption coefficient is very low. Thus its thermal radiation in that spectral region is also very low.
The answer lies in the band structure of the two materials. The band structure describes how the electrons in a solid are bound, and what other energy states are available to them.
Very simply, the band gap for transparent diamonds is very wide (see this link):
Normally, diamond is not a conductor: all the electrons live in the "valence band", and you need a photon with at least 5.4 eV of energy to push an electron into the conduction band. In the process, that photon would be absorbed. A photon with less energy cannot give its energy to an electron, because that electron "has nowhere to go". And since visible light has energies of between 1.65 and 3.1 eV, only UV photons have enough energy to be absorbed by pure diamond.
That same link also describes how impurities give rise to color in diamond: for example, nitrogen atoms produce an "intermediate" energy level, and this gives rise to more energetic electrons that could jump the gap to the conduction band and absorb light.
By contrast, graphite is a conductor. As a conductor, it has electrons in the conduction band already. You know this, because even a tiny voltage will give rise to a current - this tells us that the electrons didn't need to be "lifted" into the conduction band first. And since electrons will absorb any amount of energy easily, the material absorbs all wavelengths of light: which makes it black.
Best Answer
While I agree in principle with David Lynch's answer, I think it's good to take a closer look at the phase diagram (adapted from http://upload.wikimedia.org/wikipedia/commons/4/46/Carbon_basic_phase_diagram.png):
I added the arrows to show possible paths you might follow.
Red path: diamond would become graphite before melting; the molten carbon becomes diamond just above 10 GPa, and you cool it down while maintaining the pressure. Once the diamond is cool enough it can be depressurised slowly without changing phase (the hashed region has to be traversed carefully).
Blue path: if you just heat your diamond, it will turn to graphite and then vaporize (sublimate) around 4000 K.
The green path shows the only "sure" way to melt diamond - starting at a very high pressure, then raising the temperature; above 5000 K one could either continue raising the temperature, or lower the pressure.
Note that there is a real problem with doing this - there are no containers that I know of that will allow that combination of temperature and pressure to be maintained. Synthetic diamonds have been made, but typically not by growing from the melt...