Solved – Ratios in Regression, aka Questions on Kronmal

interactionmodelingratioregressionweighted-regression

Recently, randomly browsing questions triggered a memory of on off-hand comment from one of my professors a few years back warning about the usage of ratios in regression models. So I started reading up on this, leading eventually to Kronmal 1993.

I want to make sure that I'm correctly interpreting his suggestions on how to model these.

For a model with a ratio with the same denominator on both the dependent and independent side:
$ Z^{-1}Y = Z^{-1}1_n\beta_0 + Z^{-1}X\beta_X + \beta_Z + Z^{-1}\epsilon $
- Regress dependent ratio on the (inverse) denominator variable in addition to the other ratios
- Weight by the (inverse) denominator variable
For a model with dependent variable as a ratio:
$ Y = \beta_0 + \beta_XX + Z1_n\alpha_0 + ZX\alpha_X + Z^{-1}\epsilon $
- Regress numerator by original variables, denominator, and denominator times original variables [what about categorical variables?]
- Weight by (inverse) denominator
For model with only independent variable ratios:
$ Y = \beta_0 + X\beta_X + Z^{-1}1_n\beta_{Z^{-1}} + W\beta_W + Z^{-1}W\beta_{Z^{-1}W} + \epsilon $
- Include numerator and (inverse) denominator as main effects, ratio as interaction term.

Are my interpretations here correct?

Best Answer

You should really have linked to the Kronmal paper (and explained your notation, which is taken directly from the paper.) Your reading of the paper is too literal. Specifically, he does not give advice about weighting, rather saying that weighting can be done the usual ways, so no need to discuss. It is only mentioned as a possibility. Read your cases more like examples, especially as examples of how to analyze such situations.

In section 6 he does give some general advice, which I will cite here:

The message of this paper is that ratio variables should only be used in the context of a full linear model in which the variables that make up the ratio are included and the intercept term is also present. The common practice of using ratios for either the dependent or the independent variable in regression analysis can lead to misleading inferences, and rarely results in any gain. This practice is widespread and entrenched, however, and it may be difficult to convince some researchers that they should give up their most prized ratio or index.

The paper uses the (fictitious) example by Neyman on births and storks. To play with that example, you can access it from R by

data(stork, package="TeachingDemos")

I will leave the fun for the readers, but one interesting plot is this coplot:

Related Solutions

Weight of Evidence (WoE) in Logistic Regression – A Guide

The WoE method consists of two steps:

to split (a continuous) variable into few categories or to group (a discrete) variable into few categories (and in both cases you assume that all observations in one category have "same" effect on dependent variable)
to calculate WoE value for each category (then the original x values are replaced by the WoE values)

The WoE transformation has (at least) three positive effects:

It can transform an independent variable so that it establishes monotonic relationship to the dependent variable. Actually it does more than this - to secure monotonic relationship it would be enough to "recode" it to any ordered measure (for example 1,2,3,4...) but the WoE transformation actually orders the categories on a "logistic" scale which is natural for logistic regression
For variables with too many (sparsely populated) discrete values, these can be grouped into categories (densely populated) and the WoE can be used to express information for the whole category
The (univariate) effect of each category on dependent variable can be simply compared across categories and across variables because WoE is standardized value (for example you can compare WoE of married people to WoE of manual workers)

It also has (at least) three drawbacks:

Loss of information (variation) due to binning to few categories
It is a "univariate" measure so it does not take into account correlation between independent variables
It is easy to manipulate (overfit) the effect of variables according to how categories are created

Conventionally, the betas of the regression (where the x has been replaced by WoE) are not interpreted per se but they are multiplied with WoE to obtain a "score" (for example beta for variable "marital status" can be multiplied with WoE of "married people" group to see the score of married people; beta for variable "occupation" can be multiplied by WoE of "manual workers" to see the score of manual workers. then if you are interested in the score of married manual workers, you sum up these two score and see how much is the effect on outcome). The higher the score is, the greater is probability of an outcome equal to 1.

Solved – Interpret coefficients from logit/probit models with inverse definition of dependent or independent variable

To directly compare the values for the coefficients $\beta_1,\dots$ from studies with type (a) and (b) definition, do I just have to change the sign for $\beta$ and the corresponding t-statistic?

Yes. E.g., see the following example

set.seed(78379009)
x <- runif(100, 0, 1)
eta <- -2 + 4 * x
y1 <- 1/(1 + exp(-eta)) > runif(length(x))
y2 <- pnorm(eta) > runif(length(x))

summary(glm(y1     ~ x, binomial()))$coefficients
#R             Estimate Std. Error   z value     Pr(>|z|)
#R (Intercept) -2.200157  0.5357224 -4.106898 4.010083e-05
#R x            4.150889  0.9703970  4.277516 1.889907e-05
summary(glm(I(!y1) ~ x, binomial()))$coefficients
#R             Estimate Std. Error   z value     Pr(>|z|)
#R (Intercept)  2.200157  0.5357224  4.106898 4.010083e-05
#R x           -4.150889  0.9703970 -4.277516 1.889907e-05

summary(glm(y2     ~ x, binomial("probit")))$coefficients
#R             Estimate Std. Error   z value     Pr(>|z|)
#R (Intercept) -1.993338  0.3738787 -5.331510 9.739943e-08
#R x            4.065658  0.7045437  5.770625 7.897790e-09
summary(glm(I(!y2) ~ x, binomial("probit")))$coefficients
#R             Estimate Std. Error   z value     Pr(>|z|)
#R (Intercept)  1.993338  0.3738787  5.331510 9.739943e-08
#R x           -4.065658  0.7045437 -5.770625 7.897790e-09

Although it is $z$-stats and not $t$-stats (you are not estimating variance parameter).

How can I convert the $\beta_1,\dots$ from a study using book-to-market, such that the regression coefficient shows the marginal effect of a change in market-to-book? And how can I convert the corresponding standard errors, t-statistics, and p-values?

You cannot. The inverse of $x$ is non-linear in $x$. Also do note that it is the marginal effect on the link scale.

Best Answer

Related Solutions

Weight of Evidence (WoE) in Logistic Regression – A Guide

Solved – Interpret coefficients from logit/probit models with inverse definition of dependent or independent variable

Related Question