Solved – Coefficient of Determination with Multiple Dependent Variables

multivariate analysisr-squaredregression

I have X | Y1 | Y2 data, that I fit with some model. The model produces two values for one independent variable, where one is compared with the Y1 values, and the other is compared with the Y2 data.

To be clear: I am fitting all data at the same time, meaning I find the best parameters for the model, that describe both Y1 and Y2 as a function of X with the least overall sum of squares.

The fit works well, and now I want to calculate the R² value for the results. When I use Origin to do the fit, I get some value for R², but I have no idea how this is calculated.

I think that this is not the multiple regression case, because I have only one dependent variable. I understand how to calculate the R² value for the case where I have a single independent variable.

For example which average do I need when I build the sum? Do I use multiple average values (one for each independent variable), or do I average all Y values together?

As you may have guessed from my vocabulary, I am not very well versed in statistics, so a more Layman term description would be really great.

Edit:
Here is some example data:

A dummy model (my actual model is more complex) would be:

Y1 = A * X + B
Y2 = (A/2) * X + B

Fit results I get with Origin are:

enter image description here

Best Answer

This is a first attempt at an answer.

Source
I used your data for X, Y1, and Y2.

There is a 1:1 relationship here. A particular value of X, gives particular values of Y1 and Y2. The Y values can be thought of as a single point located in a 2d space. $Y=\left[ y_1,y_2 \right]$

Procedure:

enter the data into excel (excuse any typos)
compute the mean, slope, and intercept using normal methods
compute error between mean and actual for each row
compute error between linear fit and actual for each row
compute sum of squares for the mean-error column
compute sum of squares for the line-error column
compute the ratio of the sums in steps 5 and 6
subtract that value from 1, and compare to the provided R^2

Results from approach is shown here:

enter image description here

Compute of ratio for RSS shown here:

enter image description here

Graph of data shown here (yes, y1 label is poorly placed): enter image description here

If you have a column of error, and a mean value of the target, then you can compute a Pearson R^2 statistic.

Some relevant references:

Multivariate multiple regression in R

Related Solutions

Solved – multiple continuous independent variables, single dependent var: which analysis

You said you want "the relative contribution and co-variation among the variables." Multiple regression can give you what you're looking for, if you take advantage of what it has to offer. A correlation matrix is a very small part of what regression can yield. Assuming the predictors are relatively independent--which can be tested using tolerance or variance inflation factor statistics from a regression--the strength of the standardized coefficients will indicate the strength of each predictor's association with the outcome. T-statistics and partial and semipartial correlations can add information about this as well.

Solved – Coefficient of determination $R^{2}$ for each variable in multiple regression

The coefficient of determination is defined for the model as whole and not for individual variables. However, there is a technique called ANOVA which can roughly be thought of as breaking $R^2$ into contributions from each variable.

Recall that the coefficient of determination is defined in terms of the sums of squares of residuals:

$$ \begin{align} R^2 & = 1 - {SS_{\rm res}\over SS_{\rm tot}} \\ SS_\text{tot} & =\sum_{i=1}^n (\bar{y} - y_i)^2 \\ SS_\text{res} & =\sum_{i=1}^n (\hat{y}_i-y_i)^2 \\ \end{align}$$

Where $\hat{y}$ is the prediction vector of the model. Since we can't make a prediction $\hat{y}$ without considering all of the variables in the model.

But look at the equation for $SS_\text{tot}$ more again. This has the exact same form as the $SS_res$ if it were a trivial model with only an intercept term; such a model would predict $\hat{y}_i = \bar{y}$ for all $i$. This suggests that we are not comparing one model to some platonic ideal, but actually comparing two different models. This insight can be generalized into a chain of models:

$$ \frac{SS_1}{SS_{\text{tot}}} + \frac{SS_2}{SS_{\text{tot}}} + ... + \frac{SS_k}{SS_{\text{tot}}}= 1 $$

If we consider a chain of models, starting from the intercept only model and adding one variable at a time, then the quantity $\frac{SS_j - SS_{j-1}}{SS_\text{tot}}$ can be intrepretted as the "amount of variance explained by the $j$-th variable. As a concrete example, here is the output of the anova() function on the built-in airquality dataset:

Analysis of Variance Table

Response: Ozone
           Df Sum Sq Mean Sq F value    Pr(>F)    
Solar.R     1  14780   14780 33.9704 6.216e-08 ***
Wind        1  39969   39969 91.8680 5.243e-16 ***
Temp        1  19050   19050 43.7854 1.584e-09 ***
Month       1   1701    1701  3.9101   0.05062 .  
Day         1    619     619  1.4220   0.23576    
Residuals 105  45683     435

This is called the "sequential" analysis of variance. The Sum Sq column sums to the total sums of squares of the entire dataset, so we can see that Wind explains twice as much variance of Temp. This interpretation is subject to many caveats: it is sensitive to the order in which variables are added, and the F-scores and associated P values on the left are only meaningful for a purely linear model, etc. Nevertheless, if we take that Sum Sq column and divide by total sums of squares:

Solar.R    0.12
Wind       0.33
Temp       0.16
Month      0.01
Day        0.01
Residuals  0.38

We get a table where ever line item is roughly analogous to the quote-unquote "$R^2$" for each variable (plus one line item for the unexplained residual), although that terminology is never used, as far as I know. People talk about the proportion of variance explained instead.

Here are some additional resources if you want to read further:

Best Answer

Related Solutions

Solved – multiple continuous independent variables, single dependent var: which analysis

Solved – Coefficient of determination $R^{2}$ for each variable in multiple regression

Related Question