classificationlogisticmachine learningmulti-class
For a multi-class classification problem, we can use $K$ binary logistic classifiers, or one softmax regression classifier, so how to make the choice between the two?
IMHO, the $K$ binary logistic classifiers is just the 1-vs-all scheme for multi-class, but softmax classifier inherently handles multi-class problem. Why should I prefer one over the other?
Each hypothesis function that outputs the result of logistic regression ($h_\theta (x)$) corresponds to the probability that the inputted feature vector $\vec{x}$ is either a $1$ or a $0$ (where 1 is the 'yes'/'true'/etc. term.)
So now that you realize that each model you're creating outputs the probability that you want to know what a 'tie' means. If 2 models both output 0.4 as the probability that the inputted feature vector is part of that class, you could say that between the two the answer is indeterminate, or rather that input $\vec{x}$ is equally likely to be in 'class 1' or 'class 2' (assuming $P(y=1|x;\theta)=P(y=2|x;\theta)=\phi$ where $\phi$ is some number, which happened to be 0.4 in this example)
You can think of this visually as regressing a Bernoulli (only 2 outcomes, or regular logistic regression if you want to think of it that way) distribution and having an outcome of 0.5.
(source: sourceforge.net)
When the outcome is 0.5 from a logistic model, as with multiclass models, the inputs lie on the line of division that separates the classes, again, they are equally likely to be part of either class. This generalization is true for more classes than 2.
"Binary classification" is simply multi-class classification with 2 labels. However, several classification algorithms are designed specifically for the 2-class problem, where the response is modeled as the outcome of a Bernouilli trial.
Of course, this can be generalized to the n-dimensional setting by modeling as the outcome of a multinomial sample. For instance logistic regression can be generalized to multinomial logistic regression.
Some models, such as decision trees and ensembles of trees, can take a different form however when applied to n-class classification.
So the answer is highly dependent on the implementation of the algorithm you're using. But the baseline answer is that any multiclass classification algorithms can be used for 2-class classification, and most 2-class classification algorithms can be generalized for multi-class classification.
Best Answer
The softmax function gives a proper probability for each of the possible classes:
$$ P(y=j|x,\{w_k\}_{k=1...K}) = \frac{e^{x^\top w_j}}{\sum_{k=1}^K e^{x^\top w_k}} $$
This is nice if you want to interpret your classification problem in a probabilistic setting. Benefits of using the probabilistic formulation include being able to place priors on the parameters and obtaining a posterior distribution over classes.
That said, maybe you can imagine a really good classifier that isn't of this form. Perhaps it is of a form that is generally difficult to express (e.g. SVM -- here for multi-class details). If some such complicated classifier works well for you on a given task, perhaps you don't want to use the [potentially weaker] softmax classifier. In such a setting, there may not be a clear all-way output, so you have to settle for repeated one-vs-others classification schemes.
One more counterpoint...you could also augment the expressive power of the softmax-style approach by changing the input to the exponential. For example, it would be straightforward to replace each linear component $x^\top w_j$ with a quadratic expression $x^\top w_j + x^\top A_j x$. Other such augmentations are conceivable.