I need to compare the ability of two methods to predict an event with a binary response. Each method produces a score, where the higher score indicates 1 and a lower score indicates 0. I am looking to calculate the odds ratio (in univariable and multivariable analysis) to compare these methods. Here is the break down of method data:
Method 1: Probability between 0 and 1.
Method 2: Score between 0 and 12
In univariable analysis, the OR for method 1 will be VERY large, but for method 2 would be quite small. I think that because the two methods have different scales the odds ratios become so vastly different. The unit increase for method one should be 0.1, whereas for method two is 1.
Possible solutions are to binarize both methods, by picking a cutoff, then everything is on the 0 and 1 scale and the OR are comparable, however you loose information when you do this. Also, it can be trickey to pick an objective cutoff.
Are there any suggestions for comparing the odds ratios of continous and discrete variables?
Best Answer
Are the measures actually measuring the same thing?
For example, does a score of 0 in Method 1 match a score of 1 in Method 2, and a score of 1 match a score of 12?
If so, you can take
exp(beta*12), where beta is the regression coefficient from your logistic regression for Method 2. That would essentially give you the odds ratio not for a single step increase (1 to 2, 2 to 3, etc.) but the odds ratio for an increase from one end of the scale to the other.If instead say a 0 in Method 1 is probably closer to a 2 in Method 2, and a score of 1 is more like a 10, then you'd multiply by 10-2 = 8 instead of 12.
That should get you a more comparable interval. There are other methods you can try as well, depending on the distribution of Method 2's scores. This retains much of the information from Method 2 while still providing something approaching a like-with-like comparison.