My approach
The area of shaded region = 8×the area of 
Given that the points of the shaded region are closer to the center than the boundary of the square.
Let's talk about the boundary of the shaded region
The boundary of the shaded region therefore must be the locus of all the points whose distance from the center of the square = distance from the boundary.
Let's find the locus of the boundary of the shaded region
From the second figure
${\sqrt {h^2+k^2} } = {\sqrt {(h-h)^2 + (k-{a\over 2})^2 }}$
This simplifies to be:
$k = {a^2 – 4h^2\over 4a}$
$y = {a^2 – 4x^2\over 4a}$
Also the curve intersects the line( the hypotenuse of the triangle) y= x
For point of intersection :
$ {a^2 – 4x^2\over 4a} = x $
$4x^2 +4ax – a^2 = 0$
$x = {-4a ± \sqrt{16a^2 – 4×4×(-a^2)}\over 8}$
$x = a{(\sqrt{2} ± 1)\over 2}$
Solution 1: $x = a{(\sqrt{2} + 1)\over 2}$
Solution 2: $x = a{(\sqrt{2} – 1)\over 2}$
Solution 1 can be discarded as $ x = a{(\sqrt{2} + 1)\over 2} ≈ 1.207106 a > {a\over 2} $
Solution 2: $ a{(\sqrt{2} – 1)\over 2} ≈ 0.2071 a < {a\over 2} $
from 0 to $a{(\sqrt{2} – 1)\over 2}$ :
The curve (boundary of shaded region) lies above the line $ y=x$
so $dA = ({a^2\over 4a} – {4x^2\over 4a} – x)dx $
$dA = ({a\over 4} – {x^2\over a} – x)dx $
$\int_{0}^{A} dA = \int_{0}^{a{(\sqrt{2} – 1)\over 2}} ({a^2\over 4a} – {4x^2\over 4a} – x)dx $
A = $({a\over 4}x – {x^3\over 3a} – {x^2\over 2})]_{0}^{{a{(\sqrt{2} – 1)\over 2}}}$
$ A = {a^2\over 8}(\sqrt{2}-1) – {a^2\over 24}(\sqrt{2}-1)^3 – {a^2\over 8}
(\sqrt{2}-1)^2 $
$ = {a^2\over 8}(\sqrt{2}-1) \Biggl( 1 – {(\sqrt{2}-1)^2\over 3} – (\sqrt{2} -1)\Biggr)$
This simplifies to be equal to
${a^2\over 8}(\sqrt{2}-1)({3+5\sqrt{2}\over 3})$
$= {(7- 2\sqrt{2})\over 8×3}a^2$
Area of shaded figure = 8A
$A_{total}$ = ${(7- 2\sqrt{2})\over 3}a^2$
But the answer is :
${4\sqrt{2}-5\over 3}a^2 $
I don't know where I got it wrong, and also I have re calculated this and the result is same.
Did I miss something important or calculated wrongly
Any help of hint or suggestion or worked out solution would be appreciated.
Best Answer
Using polar coordinates
$(x, y) = (r \cos \theta, r \sin \theta)$
In the range of $ 0 \le \theta \le \dfrac{\pi}{4} $ we want
$ r \le \left( \dfrac{a}{2} - r \cos \theta \right ) $
Hence,
$ r \le \dfrac{ a } { 2 (1 + \cos \theta ) } $
The area enclosed by this polar curve
$ r(\theta) = \dfrac{ a } { 2 (1 + \cos \theta ) } =\dfrac{a}{4 \cos^2(\dfrac{\theta}{2}) } $ is given by
$A = \dfrac{1}{2} \displaystyle \int_0^\dfrac{\pi}{4} r^2(\theta) \text{d} \theta = \dfrac{1}{2} \int_0^\dfrac{\pi}{4} \dfrac{a^2}{16 \cos^4 (\dfrac{\theta}{2})} \text{d} \theta = \dfrac{a^2}{32} \int_0^{\dfrac{\pi}{4}} \sec^4(\dfrac{\theta}{2}) \text{d} \theta $
Now, let $ u = \dfrac{\theta}{2} $ then
$A = \dfrac{a^2}{16} \displaystyle \int_0^\dfrac{\pi}{8} \sec^4(u) du $
And since $\sec^2(u) = \tan^2(u) + 1 $, the above becomes
$A = \dfrac{a^2}{16} \displaystyle \int_0^\dfrac{\pi}{8} (\tan^2(u) + 1) \sec^2(u) du $
And this reduces to
$A = \dfrac{a^2}{16} ( \dfrac{t^3}{3} + t ) $
where $t = \tan(\dfrac{\pi}{8} ) = \dfrac{\sin(\frac{\pi}{4})}{\cos(\frac{\pi}{4}) + 1} = \dfrac{ 1 }{ \sqrt{2} + 1 } = \sqrt{2} - 1 $
Therefore,
$A = \dfrac{a^2}{48} ( 2 \sqrt{2} - 3(2) + 3 \sqrt{2} - 1 + 3 \sqrt{2} - 3 ) = \dfrac{a^2}{24} (4 \sqrt{2} - 5) $
And since there is $8$ of these areas in the square, then the answer is
$ \text{Area} = 8 A = \dfrac{a^2}{3} ( 4 \sqrt{2} - 5) $