As you correctly stated at the beginning, the units of $f'(x)$ are easily seen from writing the derivative as
$$ f'(x) = \frac{df}{dx} $$
so the units are the same as units of $f/x$. However, your concerns about the temperature can't be justified. The units of absolute temperature and the temperature difference are the same. In particular, the international system of units, SI, uses 1 kelvin for both.
One may also work with temperature scales that are not absolute, e.g. the Celsius degrees. The absolute zero doesn't correspond to 0 °C; in this sense, these scales are "nonlinear". However, a Celsius degree is still a unit of temperature as well as temperature difference. As a unit of temperature difference, 1 °C and 1 K are the very same thing. You can surely never forget or omit units such as degrees (of temperature) from physical quantities, whether they are computed as derivatives or not.
Similarly, adding two absolute temperatures is invalid.
It may be unnatural or useless in most physical situations (see Feynman's "Judging Books By Their Cover") but it is a valid procedure when it comes to the units. After all, absolute temperatures are just energies per degree of freedom so adding absolute temperatures isn't much different from adding energies which is clearly OK.
In particular it does not make sense to do $f'(x)\cdot y$ where $y$ is an absolute temperature.
It makes perfect sense. Thermodynamics is full of such expressions. For example consider $f(x)=S(t)$, the entropy as a function of time. Then $S'(t)\cdot T$ is a term that appears in the rate of change of some energy according to the first law of thermodynamics.
Quite generally, it is not sensible to single out temperature in these discussions. The same comments hold for distances, times, or pretty much any other physical quantities. Take time. One may consider the "current year". It's some quantity whose unit is 1 year. (Similarly, the position of something in meters.) And one may consider durations of some events whose units may also be years. It is the same unit. It is obvious that the difference $A-B$ i.e. any difference has the same units as $A$ as well as $B$. In my "current year" analogy, $Y=0$ corresponds to the birth of Jesus Christ, a random moment in the history of the Universe. That's totally analogous to $t=0$ °C, the melting point of ice. But in both cases, the time differences or temperature differences have the same units as the quantities from which the differences were calculated – such as temperature (whether they're absolute or not) or dates.
It wouldn't make any sense to have different units for quantities and their differences because dimensional analysis would cease to hold: one could no longer say, among other things, that the units of $A-B$ are the same as units of $A$ or $B$ separately.
It is very correct that one cannot calculate a sensible value of $\exp(a)$ if $a$ is dimensionful i.e. if it has some nontrivial units. Such an exponential would be adding apples and oranges, literally. Express $\exp(a)$ as the Taylor expansion, $1+a+a^2/2+a^3/6+\dots$. If $a$ fails to be dimensionless, each term has different units so it's not dimensionally correct to add them. For this reason, all exponentials (and, with a somewhat greater tolerance, logarithms) in physics are exponentials of dimensionless quantities (which have no units). The desire to avoid physically (and mathematically) meaningless quantities such as exponentials of dimensionful quantities is one of the very reasons why we use units and dimensional analysis at all. It is not a "problem"; it is a virtue and the very point of these methods.
Best Answer
One definition of work is "a change in energy." Any change in a physical quantity must have the same units as that quantity.
Different kinds of work are associated with different kinds of energy: conservative work is associated with potential energy, non-conservative work with mechanical energy, and total work with kinetic energy. In fact, that's one way to see the oft-quoted Law of Conservation of Energy:
$$ W_{total}=W_{non-conservative}+W_{conservative}\\ \Delta KE=\Delta E - \Delta PE \\ \therefore \Delta E=\Delta KE + \Delta PE $$
So just like impulse (which is a change in momentum) has the same units as momentum, work has the same units as energy. Any change in a physical quantity must have the same units as that quantity. A change in velocity has units of velocity, etc.
A more difficult question might be why torque has the same units as energy. This is more subtle, but the key concept is this: units are not the only thing that determines a quantity's interpretation. Context matters too. Energy and torque may have the same units, but they are very different things and would never be confused for one another because they appear in very different contexts.
One cannot blindly look at the units of a quantity and know what is being discussed. A dimensionful quantity might be meaningless or meaningful depending on the context, and it's meaning can change with that context. Action times speed divided by length has the same units as energy but without any meaningful interpretation (as far as I'm aware).