Celsius was defined by fixing the mercury's expansion coefficient with respect to temperature as constant. It is now defined as Kelvin plus 273.15, not the other way around. In fact, the freezing point of water under standard atmospheric temperature is 0.000089(10)°C, boiling point 99.9839°C. Mercury thermometer is now an approximate measuring device, as any measuring device, rather than the definition of Celsius scale.
The original definition of Celsius and Fahrenheit are arbitrary and artificial, but Kelvin, or thermodynamic temperature, is based on universal physical principle, i.e., second law of thermodynamics. This relation with fundamental physical principle makes Kelvin the "clean" scale of temperature.
Kelvin still depends on water, with its triple point fixed at 0.01°C (273.16K). The proposed redefinition of Kelvin will fix Boltzmann constant, the constant relating temperature and energy. That will make Kelvin even more natural.
The NIST style guide is pretty good — that's a place where people really care about getting details right.
I use lower-case names for spelled-out units, even when named for famous people or having uppercase abbreviations (N -> newton, J -> joule, L -> liter (unless you count $\ell$ for liter), K -> kelvin). I think that "degrees kelvin" is entirely by analogy with "degrees Celsius" and "degrees Fahrenheit" (the latter two of which I think I have always seen capitalized).
I don't pluralize "kelvin" when talking about temperatures: I talk about temperatures like "two hundred fifty millikelvin," or "four kelvin" for the boiling point of helium, or "three hundred kelvin" for room temperature. This is not consistent with the way that I would discuss a length unit, or a mass unit.
I think that this may be because temperature is an intensive variable. If I have a thing that weighs a kilogram, and another thing that weighs four kilograms, and I put them together, I know that I have five kilograms worth of stuff. But if I have some stuff at one kelvin, and some other stuff at four kelvins, and I mix them together, I don't get some stuff at five kelvins. I know this, and so I don't think of "a kelvin" as a lump of temperature that I can carry around and add or subtract to things.
I feel the same way about the hertz: I have no desire to say "sixty hertzes." Combining an oscillator at 60 Hz and an oscillator at 10 Hz gives me something much more complicated than an oscillator at 70 Hz. I notice that "hertz" is listed as one of your three exceptions, though.
If that's really my thought process I would make the same decision about the pascal (for pressure) and the poise (for viscosity); I can't think off the top of my head of another intensive quantity with a named unit. I think that if you asked me I would tell you that air pressure at sea level is "ten to the five pascal," but I'm focusing too hard on it and I'm honestly not sure.
In response to a comment: I definitely
do say things like "two atmospheres of pressure," but never "two bars" or "one thousand torrs." It could well be that dealing with kelvins one by one is so rare that I don't think of them as being countable. Interesting.
Best Answer
To answer this question it may help to take an example from a more familiar area of physics, and then discuss temperature.
For a long time the kilogram (the SI unit of mass) was defined as the mass of a certain object kept in a vault in Paris. Then the gram can be defined as one thousandth of the mass of that object, and so on. If you now ask, what units are being used to state the mass of the chosen object? then it does not matter as long as they are proportional to the scale of units you want to adopt. So if someone were to tell you the mass of the special object in pounds (e.g. 2.2 pounds) then you would still know that one gram is a thousandth of that.
With temperature it goes similarly. There is a certain state of water, water vapour and ice all in mutual equilibrium. That state has a temperature independent of other details such as volume, as long as the substances are pure and they are not crushed up too small. So that state has a certain temperature. It has one unit of temperature in "triple point units" (a temperature scale that I just invented). When we say the Kelvin is a certain fraction of that temperature, we are saying that a thermometer whose indications are proportional to absolute temperature must be calibrated so as to register 273.16 when it is put into equilibrium with water at the triple point, if we wish the thermometer to read in kelvin. For example, if the thermometer is based on a constant-volume ideal gas then one should make the conversion factor from pressure in the gas to indicated temperature be a number which ensures the indicated temperature is 273.16 at the triple point. You then know that your gas thermometer is giving readings in kelvin, and you never needed to know any other units. (Note, such a thermometer is very accurate over a wide range of temperature, but it cannot be used below temperatures of a few kelvin. To get to the low temperature region you would need other types of thermometer. In principle they can all be calibrated to agree where their ranges overlap.)
(Thanks to Pieter for a detail which is signaled in the comments and now corrected in the text, but I hope the comment will remain.)