In optimization point of view of generalized linear modeling, there is a transfer function that maps a linear score to a final target. There is also a loss function that is minimized in training to obtain the parameter.
Considering multinomial logistic regression, I am trying to figure out these two functions.
Are both the loss and transfer functions, considered to be -softmax in softmax regression? If not what is each of them?
Aslo is the loss function convex in parameter?
Finally is there any relation between CRFs and softmax regression models?
Best Answer
The loss function is the negative Maximum Log Conditional Likelihood(LCL) Function which also can be viewed as a cross entropy function; while the tansfer function is the softmax function.
Since the loss funciton is cross entropy, it is then convex. For detailed explanation please refer to this awesome answer.
Acturally there is. They are both log linear model and two special cases of it. For softmax regression it is features that are trained but in CRFs it is feature functions. Digging deep into this tutorial you'll know more.