First, if the other fees are paid as part of the loan, I find a payment of 138.1625. If they are paid in advance, I believe they still count as interest. Then the effective interest is 159.95. The average balance is about half of the amount borrowed, as you start off owing 1498.5 and end at 0. So the effective annual interest rate is about 159.95/(1498.5/2)=23.15%. I think they get a lower value because they do an amortization, which keeps the balance higher for longer, but it is not far off. For the shorter terms, the interest rate goes higher because the fixed 39.95 fee gets spread over fewer months.
Let's collect the comments into an answer. I like single letter variables for formulas, so
T -- $amount; or principal
n -- $numPay; number of months
R -- $payment; monthly rate
F -- $fee;
x -- $x; monthly interest rate
The payment plan, as I understand it, has an initial debt or principal $T$, and additional a fee $F$. After a month $R+F$ are paid, then every month $R$, after the n-th payment, the debt is paid in full.
As stated, the interest on the fee for the one month is not raised, which means that $F$ instead of $F\cdot(1+x_{nominal})$ is paid after a month. In balance, this still increases the effective interest rate.
The balance after $n$ months, seen from the start of the debt, and according to the flow of payments, reads as
\begin{align}T+F&=(R+F)(1+x)^{-1}+R(1+x)^{-2}+\dots+R(1+x)^{-n}\\
&=F(1+x)^{-1}+R\frac{1-(1+x)^{-n}}{x}\\
Tx+F\frac{x^2}{1+x}&=R(1-(1+x)^{-n})
\end{align}
Define
$$f(x)=Tx(1+x)+Fx^2-R(1+x-(1+x)^{1-n})$$
with derivative
$$f'(x)=T(1+2x)+2Fx-R(1-(n-1)(1+x)^{-n})$$
for a Newton iteration with the initial guess
$$x_0=\frac{nR-T}{\frac{n(n+1)}2R+F}$$
Best Answer
Let $r$ denote the APR we seek to establish, and the payments are made each $\frac{1}{n}$ of the year (i.e. $n=12$ for monthly payments, and $n=26$ for biweekly). Let $a_k$ be the principal amount of the loan at the beginning of the $k$-th period. Then $a_0 = a$, and $$ a_{k+1} = a_k (1+r)^{1/n}-p $$ Such a recurrence equation is not hard to solve using generating function technique, i.e. multiplying both sides of the equation with $x^k$ (for some indeterminate $x$) and forming $g(x) = \sum_{k=0}^\infty a_k x^k$: $$\begin{eqnarray} \sum_{k=0}^\infty a_{k+1} x^k &=& (1+r)^{1/n} \sum_{k=0}^\infty a_k x^k - p \sum_{k=0}^\infty x^k \\ \frac{f(x)-a_0}{x} &=& (1+r)^{1/n} f(x) - p \frac{1}{1-x} \\ f(x) &=& \frac{1}{(1+r)^{1/n}-1} \left( \frac{p}{1-x} - \frac{a + p - a(1+r)^{1/n}}{1 - (1+r)^{1/n} x} \right) \\ a_{k} &=& \frac{1}{(1+r)^{1/n}-1} \left( p - \left( a + p - a (1+r)^{1/n} \right) (1+r)^{k/n} \right) \end{eqnarray} $$ Requirement that after $m$ equal payments the loan is paid off gives the equation for effective annual percentage rate $r$: $$ p = \left( a + p - a (1+r)^{1/n} \right) (1+r)^{m/n} $$ As this is a non-linear equation, solving for $r$ will need to be done numerically. Here is a code in Mathematica:
If the time to the first payment is different, $a_0$ needs to be replaced with different effective principal amount, $a_0^\ast = a_0 \left(1 + r \right)^{d/n}$, where $d$ is the length of the no-payment period given as a fraction of payment intervals. For example, if payments are made monthly $n=12$, there are 4 payments to be made $m=4$ in the amount of $p=\$260.00$ on the loan of $\$1000.00$ with 1 month no payment period, the effective APR is