Eliminate $\theta$ from the equations.
$$4x=5\cos\theta -\cos 5\theta$$
$$4y=5\sin\theta -\sin 5\theta$$
Alternative forms are
$$x=5\cos^3\theta-4\cos^5\theta$$
$$y=5\sin^3\theta-4\sin^5\theta$$
and $$x=\cos^3\theta(3-2\cos2\theta)$$
$$y=\sin^3\theta(3+2\cos2\theta)$$
Best Answer
These are the parametric equations of an epicycloid, where the large circle has radius $R=1$, and the smaller exterior rolling circle has radius $r=\frac{1}{4}$.
A possible simplification is writing $$4a=4(x+i y) = 5 z - z^ 5\\ 4b=4(x-i y) = \frac{5}{z} - \frac{1}{z^5}$$ where $z = \cos \theta + i \sin \theta$ and get a relation between $a$, $b$, that implies a relation between $x$ and $y$, see link. Or, we can just give the result provided by WA :
$$-81 - 45 x^2 + 365 x^4 - 15 x^6 - 480 x^8 + 256 x^{10} - 45 y^2 - 2395 x^2 y^2 - 45 x^4 y^2 - 1920 x^6 y^2 + 1280 x^8 y^2 + 365 y^4 - 45 x^2 y^4 - 2880 x^4 y^4 + 2560 x^6 y^4 - 15 y^6 - 1920 x^2 y^6 + 2560 x^4 y^6 - 480 y^8 + 1280 x^2 y^8 + 256 y^{10}=0$$
a curve of degree $10$
$4$ leaves" />
We can simplify the defining polynomial by writing it as a polynomial in $x^2 + y^2$ and $x^2 y^2$.
$\bf{Added:}$ Following the solution of @ACB: we have the polar representation ($x = r \cos \phi$, $y=r \sin \phi$)
$$\cos^2 2 \phi = \frac{(9 - 4 r^2)(16 r^4+ 3 r^2 + 6)^2}{3125 r^4}= f(r)$$
The function in $r>0$ on RHS is strictly decreasing on $(0, \infty)$. Indeed, we have $$f'(r) = -24\, \frac{(-1 + r^2)^2 (9 + 16 r^2) (6 + 3 r^2 + 16 r^4)}{3125\, r^5}$$ Now, $f(1) = 1$, and $f(\frac{3}{2})=0$. We see that our curve is situated between the circles of radius $1$, and $\frac{3}{2} = 1 + 2\cdot \frac{1}{4}$, as it should.
So now we can write the curve in polar coordinates $$r = f^{-1}(\cos^2 2\phi)$$ where $f^{-1}\colon [0,1]\to [1, \frac{3}{2}]$ is the inverse function of $f$.