Firstly let's calculate the CDF of $X$: $$F_X(x) = P(X \le x) = \frac{\pi x^2}{\pi r^2} = x^2, 0 \le x \le 1$$ hence the PDF of $X$ is $f_X(x) = 2x, 0 \le x \le 1$ then the joint PDF of $X, Y$ is $$f(x, y) = f_X(x) f_Y(y) = 4xy, \ \ 0 \le x \le 1, 0 \le y \le 1$$
Now, compute the joint CDF of $U, V$:
$$P(min(X, Y) \le u, max(X, Y) \le v) = \\ P(min(X, Y) \le u, max(X, Y) \le v|X>Y) \times P(X>Y) + P(min(X, Y) \le u, max(X, Y) \le v|X \le Y) \times P(X \le Y) = \\ = P(Y \le u, X \le v | X <Y) \times\frac{1}{2} + P(X \le u, Y\le v|X \le Y) \times \frac{1}{2} = \\ = \frac{u^2v^2}{\frac{1}{2}} \times \frac{1}{2} + \frac{v^2u^2}{\frac{1}{2}} \times \frac{1}{2} = 2u^2v^2, 0 \le u \le v \le 1$$
Finally the joint PDF of $U, V$ is $$\frac{\partial^2}{\partial u\partial v} 2u^2v^2 = 8uv, \ 0 \le u \le v \le 1$$
Suppose rays are drawn from the center of the disk to the places where the first two darts landed. And suppose for example the angle between those two rays is $23^\circ.$ Say you rotate the ray to the first dart $23^\circ$ counterclockwise to get the ray to the second dart. If you go counterclockwise from that ray $180^\circ$ you will sweep out a part of the disk within which, if the third dart lands there, then all three will be on the same side of that line formed from that ray and the ray $180^\circ$ counterclockwise from it. But also, if you start with the ray to the second dart and go $180^\circ$ clockwise you will sweep out a region within which, if the thrid dart lands there, then all three will be in that half of the disk.
Thus you will fail to put all three on the same side of some line through the center only if the third dart lands in the region $23^\circ$ wide that is opposite the one bounded by the the two aforementioned rays. Thus the probability that they would be in a common half of the disk is
$$
\frac{360^\circ-23^\circ}{360^\circ}.
$$
All of that holds if the angle is $23^\circ.$ But the angle is uniformly distributed between $0^\circ$ and $180^\circ,$ so what you need is the expected value of
$$
\frac{360^\circ - (\text{random angle})}{360^\circ}.
$$
Since an angle uniformly distributed between $0^\circ$ and $180^\circ$ is on average $90^\circ,$ the random quantity above is on average
$$
\frac{360^\circ-90^\circ}{360^\circ} = \frac 3 4.
$$
Best Answer
Your thought process looks good, but the integration is actually a bit tricky. Here is your dartboard:
We'd like to calculate the area of the center shape, and as you say, we can split this into eight slices of equal area. Here we see the two slices in the first quadrant:
Let's compute the area of the slice from $45^{\circ}$ to $90^{\circ}$:
$$\int_{0}^{\sqrt{2}-1}(\frac{1-x^2}{2}-x)dx=\frac{1}{6}(4\sqrt{2}-5)$$
Multiplying this by $8$ gives us the area of the center shape. Then, we divide by $4$, the area of the dartboard. The final probability is then:
$$\frac{1}{3}(4\sqrt{2}-5)\approx.2190$$
Judging by our picture above, this seems right. The center shape is about $1/5$ the area of the dartboard.