In a multi-class classification problem, models that I've trained (and which I intend to use in a majority voting ensemble) consistently get weak performance on a couple of specific classes. There is class imbalance, but the "bad" classes are not the least frequent classes. What strategies are there to boost performance specifically for those classes?
I had an idea but I'm not quite sure how to implement it: train an additional model that does binary classification on the weak class vs all others, then add that to the majority vote (either raw votes or probabilities) ensemble to give stronger weight to that class when other models get it wrong. Is this a reasonable approach?
Best Answer
I'd say the most straightforward way is to modify your loss function to weight errors on the "bad" classes stronger in relation to other errors. Of course this is a trade-off at the cost of lost performance somewhere else. You generally cannot just get better performance on these classes for nothing.
For underrepresented classes I'd have recommended to oversample them, but according to your description that does not seem to be the issue.