The definition of the cumulative distribution function of a random variable $X$ is the function given by $\displaystyle F_X(x)=P(X \leq x)$.
If $Z=X+Y$,
$\displaystyle F_{Z}(z)=P\{X + Y \leq z\} = \int_{0}^{z}\int_{0}^{z-x}2\,\mathrm dy\,\mathrm dx=2\int_{0}^{z}(z-x)\mathrm dx=z^2$
(Rather than integrating, it is easy to determine the CDF $F_{Z}(z)$ by
inspection:)
The PDF is the derivative of the CDF;
$\displaystyle \Rightarrow f_{Z}(z)=\frac{d}{dz}F_{Z}(z)=2z$.
If $Z=X-Y$,
$\displaystyle F_{Z}(z)=P\{X - Y \leq z\} =\mathbb P(Y\geq X-z)$
$\displaystyle \color{red}A\color{red}R\color{red}E\color{red}A\color{black} =\frac{1}{2}\left(\frac{1-z}{\sqrt{2}}\right)^2=\frac{(1-z)^2}{4}$
$\displaystyle \color{blue} A\color{blue}R\color{blue}E\color{blue}A\color{black} = \frac{1}{2}-\color{red}A\color{red}R\color{red}E\color{red}A\color{black}=\frac{1}{2}-\frac{(1-z)^2}{4}$
Therefore $\displaystyle F_{Z}(z)=2\left[\frac{1}{2}-\frac{(1-z)^2}{4}\right]=\frac{1}{2}(-z^2+2z+1)$
And differentiating to find the PDF:
$\displaystyle f_{Z}(z)=\frac{d}{dz}\left(\frac{1}{2}(-z^2+2z+1)\right)=1-z$.
If $Z=XY$,
$\displaystyle F_{Z}(z)=P\{XY \leq z\} =\mathbb P\left(y\leq \frac{z}{x}\right)$
The points of intersection of $\displaystyle y=\frac{z}{x}$ with the line $\displaystyle y=-x+1$ are:
$\displaystyle x=\frac{1\pm \sqrt{1-4z}}{2}$
The curve intersects the triangle if $z<\frac{1}{4}$.
$\color{blue}A\color{blue}R\color{blue}E\color{blue}A \color{black}=$QUADRILATERAL$+$INTEGRAL$+$TRIANGLE$ $
$\displaystyle=\frac{1}{4}\left(2z-\sqrt{1-4z}+1\right) + \int_{\frac{1-\sqrt{1-4z}}{2}}^{\frac{1+\sqrt{1-4z}}{2}}\frac{z}{x}dx+\frac{1}{8}\left(\sqrt{1-4z}-1\right)^2$
$\displaystyle =\frac{1}{4}\left(2z-\sqrt{1-4z}+1\right) + 2z\tan^{-1}\sqrt{1-4z} +\frac{1}{8}\left(\sqrt{1-4z}-1\right)^2$
$\displaystyle = \frac{1}{2}\left(1-\sqrt{1-4z}+4z\tan^{-1}\left(\sqrt{1-4z}\right)\right)$
Therefore, multiplying by 2, we have: $F_{Z}(z)=\left(1-\sqrt{1-4z}+4z\tan^{-1}\left(\sqrt{1-4z}\right)\right)$
And taking the derivative to find the PDF, we have:
PDF $=\frac{d}{dz}\left(1-\sqrt{1-4z}+4z\tan^{-1}\left(\sqrt{1-4z}\right)\right)$
$\displaystyle=4\tan^{-1}\left(\sqrt{1-4z}\right)$.
Further exercises related to the question:
The marginal distribution for $X$ is:
$\displaystyle f_{X}(x)=\int_{-\infty}^{\infty}f_{X,Y}(x,y')dy'=\int_{0}^{1-x}2dy=2(1-x)$
The marginal distribution for $Y$:
$\displaystyle f_{Y}(y)=\int_{-\infty}^{\infty}f_{X,Y}(x',y)dx'=\int_{0}^{1-y}2dx=2(1-y)$
The conditional PDF of $X$ given $Y$:
$\displaystyle f_{X|Y}(x|y)=\frac{f_{X,Y}(x,y)}{f_{Y}(y)}=\frac{1}{1-y}$, $0\leq x \leq y$
Since the conditional PDF is uniform on $[0,1-Y]$, the conditional
expectation is simply: $\displaystyle\mathbb E[X|Y=y]=\frac{1-y}{2}$
The total expectation theorem yields:
$\displaystyle\mathbb E[X]=\int_{0}^{1}\frac{1-y}{2}f_{Y}(y)dy=\frac{1}{2}\int_{0}^{1}f_{Y}(y)dy-\frac{1}{2}\int_{0}^{1}yf_{Y}(y)dy$
$\displaystyle \mathbb E[X]=\frac{1}{2}-\frac{1}{2}\mathbb E[Y]$ and by symmetry, $\mathbb E[X]=\mathbb E[Y]\Rightarrow$
$\displaystyle\mathbb E[X]=\frac{1}{2}-\frac{1}{2}\mathbb E[X]\Rightarrow \mathbb E[X]=\frac{1}{3}$. $\blacksquare$
Best Answer
We know that $f_{XY} (x,y)$ is constant inside the triangle since $(X,Y)$ is uniformly chosen. Then the integral of the p.d.f. over the triangle $T$ with base $b$ and height $h$ reduces like this:
$$\iint \limits_Tf_{XY} dA = bh/2 \cdot f_{XY} = 1$$
And since $b = h =1$, $f_{XY}=2$ inside the triangle.