Solved – ny guideline for interpreting a hazard ratio as ‘small’, ‘medium’ or ‘large’

cox-modelhazardsurvival

I would appreciate if anyone can share a reference that discusses magnitudes of hazard ratios as effect sizes. Specific subject matter obviously weighs in when trying to determine what a relevant effect size is, but the problem I have seen in some recent papers is that time-to-event Cox PH analyses with large sample sizes (i.e., in the thousands, that's a large sample in health sciences) result in some arguably tiny hazard ratios labeled 'significant' and the authors take that as evidence of relevance or consequence. The large sample size fallacy is nothing new, but it's difficult to argue about magnitude of an effect when there is no general frame of reference. For instance, I am aware that an odds ratio can be converted into a standardized mean difference (in the log odds scale) as shown in this paper http://www.ncbi.nlm.nih.gov/pubmed/11113947 , for which, thanks to Cohen (1988), there are some generally agreed magnitudes for 'small', 'medium' and 'large' (at least in the social and behavioral sciences). Not that these magnitude guidelines are set in stone, but they do have some justification, and Cohen explains they are just general qualitative definitions.

Cohen J. (1988). Statistical Power Analysis for the Behavioral Sciences, 2nd Ed. Hillsdale, NJ: Laurence Erlbaum Associates

Best Answer

Cohen's approach been increasingly criticized. In general, attempting to define cutoffs for "small", "large", etc. is futile. The interpretation of any statistical measure is context-dependent.

What is useful is to supplement a relative effect (hazard ratio, odds ratio, etc.) with an absolute effect. Suppose one had a model with only age and sex as predictors. An absolute effect estimate might be the difference in estimated 5-year survival probabilities for males at age x with that of females at age x, where x is some convenient value such as the median age in the combined samples.

Related Question