Regression coefficient on a triangle using geometry

geometrymachine learningmathematical-statisticsregressionregression coefficients

I am encountering a question as follows:

Let $X, Y$ be two independent uniform random variable on $(0,1)$. We consider the regression model $Y = \beta_1 X + \beta_0$, given the restriction that $X + Y > 1$. We want to find $\beta_1$.

Let us call $U = X|X + Y > 1$, $V = Y|X+Y > 1$. Then we could compute the density of $U$ and $V$ by doing conditional probability, and $\beta_1 = \frac{Cov(U,V)}{Var(U)}$, $\beta_0$ could be found by plugging in the mean of $U$ and $V$. The final answer should be $\beta_1 = -\frac{1}{2}$.

My question is: how to compute $\beta_1$ based on geometry? If we draw the graph, we could see that we are doing a right-top triangle within the unit square $[0,1] \times [0,1]$. We could also know the center of the triangle is $(\frac{2}{3}, \frac{2}{3}) = (E[U], E[V])$. Is there a way that we could visualize the $\beta_1$ should be $-\frac{1}{2}$?

Best Answer

Because $(X,Y)$ has a uniform distribution over the triangle shown, the expectation of $Y$ conditional on $X$ evidently splits the lower and upper boundaries of the triangle, shown as the dotted line $y = 1 - x/2:$

enter image description here

That's the regression of $Y$ on $X.$ Because it happens to be a linear function, it's also the (Ordinary Least Squares, or "OLS") linear regression.


We can prove this from first principles. The density function (supported in the blue triangle of the diagram) is

$$f_{X,Y}(x,y) = 2\mathcal I(0\le x\le 1,\ 1-x \le y\le 1).$$

We will need the first and second moments, as always, so let's calculate them now. By symmetry $X$ and $Y$ have the same expectation,

$$E[Y] = E[X] = \iint x f_{X,Y}(x,y)\,\mathrm d x\, \mathrm d y = 2 \int_0^1\int_{1-x}^1 x \,\mathrm d x\, \mathrm d y = \frac{2}{3}.$$

Similarly

$$E[Y^2] = E[X^2] = \frac{1}{2}$$

and

$$E[XY] = \frac{5}{12}.$$

The least squares objective is the average squared deviation between $Y$ and $\alpha + \beta X$ for unknown parameters to be determined:

$$\Lambda = \iint (y - (\alpha + \beta x))^2\,f_{X,Y}(x,y)\,\mathrm d x\,\mathrm dy.$$

This is a differentiable function of the parameters (which can be any real numbers) and the differentiation can be carried out under the integral sign, telling us that

$$\Lambda_\alpha = 2\iint y - (\alpha + \beta x)\,f_{X,Y}(x,y)\,\mathrm dx\,\mathrm dy = 2\left(E[Y] - \alpha - \beta E[X]\right)$$

and

$$\Lambda_\beta = 2\iint x(y - (\alpha + \beta x))\,f_{X,Y}(x,y)\,\mathrm dx\,\mathrm dy = 2\left(E[XY] - \alpha E[X] - \beta E[X^2]\right).$$

Equating both with zero gives all possible critical points. Plugging in the expectations computed previously gives

$$0 = 2\left(\frac{2}{3} - \alpha - \beta \frac{2}{3}\right)$$

and

$$0 = 2\left(\frac{5}{12} - \frac{2}{3} - \beta \frac{1}{2}\right).$$

This system of linear equations (the Normal equations of OLS) has the unique solution (easily found)

$$(\alpha,\beta) = (1, -1/2),$$

as we saw in the diagram.

Related Question