Is it possible to compare probabilities of 2 logistic different models? For example if I have one model that returns the probability that someone answer a phone call on Mondays, and then I have another model for Tuesday, and another for Wednesday and so on…
Then for the same input I run the first model and I get that the probability for that person of being contacted is .8 while for the model of Tuesdays is .6 and the for the rest of the days is also less than 0.8.
Would it be ok to compare those, and say that is more probable to contact this person on Tuesdays or those probabilities are not comparable?
I think that they are not because those models might have for example different error rates.
If this is the case, how would you do a model that gives you do the best time to contact someone?
I would really appreciate some light in this subject. thanks
Best Answer
Indeed, you cannot reliably compare across logit models with different underlying data. Without repeating what has been written before, this post has a very good answer (or see this paper).
In your case, combine the data from different days, and model this:
$answer=\alpha+\beta_1Tues+\beta_2Wed+\beta_3Thurs+\beta_4Fri+\beta_5Sat+\beta_6Sun$
You can do simple Wald tests or likelihood ratio tests to compare whether the coefficients for each day are statistically different. You may find, for example, that there is no statistical difference between Sat and Sun, in which case you could update your model:
$answer=\alpha+\beta_1Tues+\beta_2Wed+\beta_3Thurs+\beta_4Fri+\beta_5Weekend$
You can also estimate the marginal effects of each day, as odds ratios can be confusing or misleading depending on what you are really interested in.
If you have time of day, that can be a multiplying effect, which may moderate the day, though interpreting interaction terms in logit models can be confusing.
In addition, other variables may mediate the effect of the specific day - employment status, marital and parental status, etc. If you have these you may want to include them as controls.