Firstly, to remove any doubts, you are correct so far in that:
- The radius of the circle is $10$ (though you kind of went about this in a roundabout method)
- The diagonal is $10\sqrt{2}$
- The square has area $100$
- The area of each quarter-circle is $25\pi$ for a total of $50\pi$
However, you forget that they overlap. This means that while the quarter circles sum to $50\pi$ in area, some of it just doesn't matter, since the other circle is already covering up that bit of space. (If you've ever heard of the inclusion-exclusion principle for stuff like probabilities and counting problems, it has a very similar feel to it.) Consider the below picture, which only has one of the circles in it. Imagine rotating a copy of this square $180^\circ$ and superimposing it on itself: you obviously end up with the original picture you have, but with overlap:
In fact, it is the area of the overlap you are tasked to find. That overlap is the blue area in the picture below:
With the picture color-coded like this, it should not be difficult to convince you that
$$\text{(The blue area)} = \text{(The square's area)} - \text{(The two bits of green area)}$$
In a similar vein, the first picture should show you that the area of one portion of the green area can be obtained by taking the square, and subtracting the area of the quarter circle. Double that for the green area, and subtract that from the area of the square. Then you have your result, the area of the overlapping region.
Find area $S$ first:
$$S=\frac16 a^2\pi-P_{\triangle ADK}$$
Area of ADK is:
$$P_{ADK}=2S+P_{\triangle ADK}=2(\frac16 a^2\pi-P_{\triangle ADK})+P_{\triangle ADK}=\frac13 a^2\pi-P_{\triangle ADK}$$
$$P_{ADK}=\frac13 a^2\pi-\frac14a^2\sqrt3$$
Shaded area is simply:
$$P_{shaded}=P_{ADC}-P_{ADK}=\frac14 a^2\pi-(\frac13 a^2\pi-\frac14a^2\sqrt3)$$
$$P_{shaded}=\frac14a^2\sqrt3-\frac1{12}a^2\pi=\frac1{12}a^2(3\sqrt3-\pi)$$
Best Answer
That is the picture that fits the problem.
See that
$$CE=\sqrt{2}=CA_1+A_1A+AE=\sqrt{2}r+2r+\sqrt{2}r \to r=\frac{\sqrt{2}}{2+2\sqrt{2}}=\frac{2-\sqrt{2}}{2}$$
EDIT
Hint
To prove that it is the maximum work with the picture below:
Work with variation of $\alpha$, the trapezium $EFGK$ and $DG+GK+KC=DC=1$.