Are there any professional alternatives to TeX/LaTeX which produce equal or even better typesetting? Maybe expensive commercial ones. The software counts if it is comparable or superior to LaTeX (with microtype) in typesetting text or math (or both). In any case, please mention both aspects in your answer.
I am only aware of ConTeXt (which is partly based on TeX).
Note that I am not looking for an alternative for my personal use, I am just asking out of curiosity.
You asked for professional solutions. Several top technical journals, including I believe Physical Review Letters and Nature magazine, use Advanced Print Publisher (APP), also known as Advent 3B2. Formerly available from Arbortext but now from PTC.
It can definitely handle mathematics, and I'm sure it deals with typographic details such as ligatures and microtype-like tweaks. I believe the main strength over tex is that it makes complete document production very automated. Things like dealing with floats (tables and figures) in a more-automatic way than tex, and giving manual controls to override it.
I haven't used it, however. (standard desktop version starts at US$5000-10000 as of January 2015). The wikipedia page has a little information.