Suggested approach: Prove that $TF \times TX = TB \times TA$, where lines $DE$ and $AB$ intersect at point $T$. (In particular, independent of the $P,Q$, as suggested by the problem.)
Corollary: It follows that $TF \times TX = TB \times TA = TP \times TQ$ and thus $F,X,P,Q$ are concyclic as desired.
Proof of approach: One way to prove the equation is by side length chasing. Apply Menelaus on triangle $ABC$ to transversal $TDE$ to obtain $TA/TB$ and hence $TA, TB$. Then we can find $TF, TX$ and multiply it out.
Details of side length chasing
$\frac{AT}{TB} \times \frac{BD}{DC} \times \frac{CE}{EA} = 1 $
$ \frac{AT} {TB} = \frac{ EA}{BD} = \frac{c+b-a}{c+a-b}$
$AT - TB = c \Rightarrow AT = \frac{ c (c+b-a) } { 2(b-a) } , TB = \frac{ c(c+a - b ) } { 2(b-a)}$
$TF = TB + BF = \frac{ c(c+a - b ) } { 2(b-a)} + \frac{c+a - b}{2} = \frac{ ((c-b+a)(c+a-b) } { 2(b-a)} $
$TX = TB + BX = \frac{ c(c+a - b ) } { 2(b-a)} + \frac{c}{2} = \frac{ c(c)}{ 2(b-a)}$
Now, we multiply these terms to show that $TA \times TB = \frac{ c^2 (c+b-a)(c+a-b) } { 4(b-a)^2} = TX \times TF$
Additional observations, which I couldn't use directly
$A, F, T, B$ are harmonic conjugates. This can be shown either from
1) $ \frac{ TA}{TB} = \frac{EA}{BD} = \frac{AF}{FB}$, or also from
2) Lines $AD, BE, CF$ are concurrent (at the Gergonne point)
COMMENT:
You correctly found that $\triangle IDE≈\triangle AFM$.We rewrite the required relation as:
$\frac{AF}{AM}=\frac{ID}{IE}$
That is triangles IAE and DIE must be similar. These two triangles have common angle $\angle IED$, therefore we must have:
$\angle IDE=\angle EIA$
This is not possible because:
$\angle IDE=\angle BDE (=90^o)+\angle BDI$
$\angle EIA=\angle EIF (≠90^o)+\angle FIA(=BDI)$
Best Answer
To solve this problem we need to draw two auxiliary lines, i.e. the angle bisectors of $\measuredangle A$ and $\measuredangle C$ as shown in the diagram. Both these lines passes through the incenter $I$ of the triangle $ABC$. By the way, $Q$ is the intersection point of side $BC$ and line $MN$, which is parallel to the side $CA$. Consequently, we can state that the two triangles $API$ and $CIQ$ are isosceles triangles.
It is given $AB=30$, $MP=8$, and $PN=25$. Let $AP=x$, $CA=3y$, and $CQ=z$.
You have already found one of the equations we need to solve the problem, i.e. $$30x-x^2=200 \tag{1}$$ Equation (1) has two roots, i.e. $x=10$ and $x=20$. First, we take the former and check whether it leads us to a solution. So, from now on, we have $AP=x=10$, which makes $PB=20$. As consequence, we have $AP:PB=1:2$. Since $PQ$ is parallel to $CA$, $CQ:QB=1:2$ as well.
Since $API$ is an isosceles triangle, we have $IP=AP=10$. In a similar vein, since $CIQ$ is an isosceles triangle, we can deduce $IQ=CQ=z$. Since the two lines $AC$ and $PQ$ are parallel to each other and $AP:PB=1:2$, we can write $PQ=2y$. Therefore, we are able to form the following equation. $$PQ=PI+IQ \qquad\rightarrow\qquad 2y=10+z \tag{2}$$
Now, intersecting chord theorem can be used to obtain the following equation. $$MQ\cdot QN=\left(MP+PQ\right)\left(PN-PQ\right)=CQ\cdot QB \quad\rightarrow\quad \left(8+2y\right)\left(25-2y\right)=z\cdot 2z $$ $$200+34y-4y^2=2z^2 \tag{3}$$
When we substitute the value of $z$ from the equation (2) in equation (3), we get, $$200+34y-4y^2=2\left(2y-10\right)^2=8y^2+200-80y \qquad\rightarrow\qquad 12y^2-114y=0 \tag{4}$$ Equation (4) has two roots, i.e. $y=0$ and $y=9.5$, where the latter is the only acceptable solution. When we insert this value of $y$ into the equation (2), we get $z=9$.
Therefore, the the length of the two sides $CA$ and $BC$ can be written as $$CA=3y=3\times 9.5=28.5\quad \mathrm{and}$$ $$BC=3z=3\times 9=27.$$
Now, it is up to you to work out the case where $x=20$.