Let's do it.
$$M_{XY}(t_1,t_2)=E[e^{t_1 X + t_2Y }]=\int \int e^{t_1 x}e^{t_2 y } \frac{2}{5}(2x+3y) dx dy$$
It's too hot here in the south, I'll use Maxima:
f(x,y):=(2*x+3*y)*(2/5);
ratsimp(
(4/5)*(%e^t1/t1-(%e^t1-1)/t1^2)*((%e^t2-1)/t2) +
(6/5)*(%e^t2/t2-(%e^t2-1)/t2^2)*((%e^t1-1)/t1) -
integrate(integrate(f(x,y) *exp(t1*x)*exp(t2*y), x, 0, 1),y,0,1)
);
0
So it's ok.
The title also asks about computing $E(XY)$ from the $M_{XY}$
You can do that by applying $$\frac{\partial^2 M_{XY}(t_1,t_2)}{\partial t_1 \partial t_2} \biggr\rvert_{t_1=0,t_2=0}=E(XY)$$
but 1) you must be sure that you are allowed to do that (the MGF is well behaved around zero) 2) you might consider it too cumbersome
In this case, it might appear that 1) does not apply - the MGF seems to blow up at zero, but that's not so, there is an indeterminancy that we can resolve by Taylor expansion. Because $e^x=1+x+x^2/2 +x^3/6+O(x^4)$ we have:
$$\frac{e^{x}-1}{x}=1+\frac{1}{2}x+O(x^2)$$
and
$$\frac{e^{x}}{x}-\frac{e^{x}-1}{x^2}=\frac{x+x^2+x^3/2- (x+x^2/2+x^3/6) + O(x^4)}{x^2}=\frac{1}{2}+\frac{1}{3}x+O(x^2)$$
Then, disregarding $O(x^2)$ terms:
$$ M_{XY}(t_1,t_2) =
\frac{4}{5}(\frac{1}{2}+\frac{1}{3}t_1)(1+\frac{1}{2}t_2)+
\frac{6}{5}(\frac{1}{2}+\frac{1}{3}t_2)(1+\frac{1}{2}t_1) + O(t_1^2)O(t_2^2)
$$
(to check: $M_{XY}(0,0)=1$)
Can you follow on from here? I get $E(XY)=1/3$
(Of course, it would have been easier to compute $E(XY)$ directly by integration, but I guess that's not the point).
Let $Z$ be a discrete random variable taking values $1\dots,n$ with respective probabilities $p_1,\dots,p_n$ (of course $p_1+\dots+p_n =1$).
Define $X$ and $Y$ as follows: let $x_i,y_i,~i=1,\dots,n$ be a sequence of numbers, and let $X=x_i,Y=y_i$ whenever $Z=i$.
Note then that $X<Y$ if and only if $Z$ is equal to an $i$ for which $x_i<y_i$.
The MGF of $(X,Y)$ is easily obtained by conditioning on $Z$:
$$E[e^{t_1 X+ t_2 Y} ] = \sum_{i=1}^n p_i e^{t_1 x_i + t_2 y_i}.$$
In our case $n=4$, $p_1=\frac 12=\frac{6}{12},p_2=\frac 14=\frac{3}{12},p_3=\frac 1{12},p_4=\frac {1}{6}=\frac{2}{12}$, and $x_1=1,y_1=1$, $x_2=2,y_2=1$, $x_3=0,y_3=1$ and $x_4=4,y_3=3$.
The only value of $i$ such that $x_i<y_i$ is $i=3$. Since $p_3=\frac{1}{12}$, it follows that $P(X<Y) = p_3= \frac{1}{12}$.
Can you see what is $P(X=Y)$ and $P(Y<X)$ ?
Best Answer
$$M(t_1, t_2) = E(e^{t_1 X + t_2 Y})$$
$$M(t_1, 0) = E(e^{t_1X}) = \frac{1}{2}e^{t_1} + \frac{1}{4}e^{2t_1} + \frac{1}{12} + \frac{1}{6}e^{4t_1} = M^{'}_{X}(t_1)$$
$$E(X^n) = \frac{\partial^n M^{'}_{X}(t_1)}{\partial t_1^n} \bigg\vert_{t_1=0}$$
$$\frac{\partial M^{'}_{X}(t_1)}{\partial t_1} = \frac{1}{2}e^{t_1} + \frac{2}{4}e^{2t_1} + \frac{4}{6}e^{4t_1}$$
$$\frac{\partial^2 M^{'}_{X}(t_1)}{\partial t_1^2} = \frac{1}{2}e^{t_1} + \frac{4}{4}e^{2t_1} + \frac{16}{6}e^{4t_1}$$
$$Var(X) = E(X^2)-E(X)^2 = (1/2 + 4/4 + 16/6) - (1/2 + 2/4 + 4/6)^2 = 25/18$$