Solved – Does there exist zero-inflated linear regression

regressionzero inflation

I have a non-count data with huge number of zeros in the target variable. I need to fit a model being a mixture of Dirac delta function and normal distribution parametrized by mean $X\beta$ and variance $\sigma^2$, with mixing proportion $\pi$, i.e.

$$ y \sim \left\{ \begin{array}{cl} 0 & \text{ with probability }\pi
\\
\mathcal{N}\left(X \beta, \sigma^2 \right) & \text{ with probability } 1-\pi\end{array} \right.$$

to account for the excess zeros. Could you provide me with any references about such models? Or maybe there is some approach that is better, then the above, for continuous, zero-inflated data?

Best Answer

I have found 2 references so far using zero-inflated normal regression, one in medical research and the other in animal conservation:

Semiparametric zero-inflated modeling in multi-ethnic study of atherosclerosis (mesa): note that they use $\log(y+1)$-transform: $f(y) = \log(y+1)$ , so it could be problematic (e.g., here and here).
A drop in immigration results in the extinction of a local woodchat shrike population: it looks like Bayesian models.

Both the response variables, Agatston scores of CAC and the number of fledglings of brood, are probably non-negative, however.

Related Solutions

Solved – Fitting a probability distribution to zero inflated data in R

You can use Vuong test in pscl package to compare non-nested models. Here is an example

> m1 <- zeroinfl(i.vec ~ 1 | 1, dist = "negbin")
> summary(m1)

Call:
zeroinfl(formula = i.vec ~ 1 | 1, dist = "negbin")

Pearson residuals:
    Min      1Q  Median      3Q     Max 
-0.3730 -0.3730 -0.3730 -0.2503  7.3544 

Count model coefficients (negbin with log link):
            Estimate Std. Error z value Pr(>|z|)    
(Intercept)   1.1122     0.3831   2.903  0.00369 ** 
Log(theta)   -1.9256     0.2839  -6.784 1.17e-11 ***

Zero-inflation model coefficients (binomial with logit link):
            Estimate Std. Error z value Pr(>|z|)
(Intercept)   -9.815     96.462  -0.102    0.919
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 

Theta = 0.1458 
Number of iterations in BFGS optimization: 579 
Log-likelihood: -80.51 on 3 Df


> m2 <- zeroinfl(i.vec ~ 1 | 1, dist = "poisson")
> summary(m2)

Call:
zeroinfl(formula = i.vec ~ 1 | 1, dist = "poisson")

Pearson residuals:
    Min      1Q  Median      3Q     Max 
-0.7242 -0.7242 -0.7242 -0.4860 14.2795 

Count model coefficients (poisson with log link):
            Estimate Std. Error z value Pr(>|z|)    
(Intercept)  2.05911    0.08205    25.1   <2e-16 ***

Zero-inflation model coefficients (binomial with logit link):
            Estimate Std. Error z value Pr(>|z|)
(Intercept)   0.4561     0.2933   1.555     0.12
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 

Number of iterations in BFGS optimization: 11 
Log-likelihood: -233.7 on 2 Df


> vuong(m1, m2)
Vuong Non-Nested Hypothesis Test-Statistic: 1.946095 
(test-statistic is asymptotically distributed N(0,1) under the
 null that the models are indistinguishible)
in this case:
model1 > model2, with p-value 0.02582165

Vuong test also suggests that the zero-inflated negative binomial provides a better fit to your data compared to the ordinary negative binomial (not shown here, but you can fit both models and compare them).

Solved – Modelling zero-inflated proportion data in R using GAMLSS

The answer below references the inflated beta GAMLSS documentation (Rigby & Stasinopoulos, 2010, section 10.8.2, page 215). It would seem that your data could be fitted with the inflated beta model.

The response variable for the $\nu$ component of the model is a ratio of probabilities (an odds) given by

$\nu = p_0 / (1-p_0-p_1)$

where $p_0$ is the probability of zero response and $p_1$ the probability of one response. Hence $\nu$ can take values > 0.

The $\tau$ component is given by

$\tau = p_1 / (1-p_0-p_1)$

Using the predicted response values for the $\nu$ and $\tau$ components, the probability of zero response can be computed as

$p_0 = \nu / (1+\nu+\tau)$

Hope this helps.

Best Answer

Related Solutions

Solved – Fitting a probability distribution to zero inflated data in R

Solved – Modelling zero-inflated proportion data in R using GAMLSS

Related Question