The answer to your question is, that it isn't necessarily true that $K_F^p$ is a perfect field. For example, let $F$ be any non-perfect field (e.g. $\mathbb{F}_p(T)$), and let $K=F$, so that we must have $K_F^p=F$.
When $K=\bar{F}$, an algebraic closure of $F$, then $K_F^p$ is the smallest perfect subfield of $K$ containing $F$, hence the name "perfect closure". This is Lemma 3.16 in Karpilovsky's Field Theory (for some reason I couldn't find this in a more standard reference like Lang). As you mention, $K_F^p$ consists of those $\alpha\in K$ such that $\alpha^{p^n}\in F$ for some $n$, and note that the field that Karpilovsky refers to as the perfect closure is
$$F^{p^{-\infty}}=\{a\in\bar{F}\mid a^{p^n}\in F \text{ for some }n\geq0\}.$$
First, $L/F$ is a finite and separable extension, thus by the primitive element theorem $L = F(\alpha)$ for some $\alpha \in L$. Let $m_\alpha \in F[X]$ be the minimal polynomial of $\alpha$. Since $K/F$ is Galois, over $K$, $m_\alpha$ splits into irreducible factors of the same degree, $d$ say. In Dummit-Foote this last statement is Exercise 14.4.4, the exercise right before the one in the question. Since $d$ divides $p^\ell = [L:F]$, it is a power of $p$ itself. This proves
Claim 1. For any root $\beta \in M$ of $m_\alpha$, $K(\beta)/K$ has degree a power of $p$.
Now the Galois closure $M$ is the composite field of the $K(\beta_i)$ where $\beta_i$ runs through the roots of $m_\alpha$. Reason: The composite field is the splitting field of $m_\alpha$, so it is Galois and contains $M$. On the other hand $M$ contains the splitting field of $m_\alpha$ because it contains the root $\alpha$.
Claim 2. $K(\beta)/K$ is Galois for every root $\beta$ of $m_\alpha$.
Proof. $L/K = K(\alpha)/K$ is Galois. Let $\phi : K(\alpha) \to K(\beta)$ be the isomorphism induced by $\alpha \mapsto \beta$ and the identity on $K$. Then $\text{Aut}(K(\alpha)/K) \cong \text{Aut}(K(\beta)/K)$ via $\sigma \mapsto \phi\sigma\phi^{-1}$.
Finally, the Galois group of $M$ (the composite of the $K(\beta)$) over $K$, is a subgroup of the direct product of the Galois groups of $K(\beta)/K$, which are all $p$-groups by Claim 1. Therefore it is a p-group by Dummit-Foote Proposition 14.12. Therefore $[M:K]$ is a power of $p$ and since $[K:F]$ is a power of $p$ and the same follows for $[M:F]$ by the tower formula.
Best Answer
Suppose that $F$ is not perfect; then we have $q:=\operatorname{char}F>0$, and there exists an element $\alpha\in F$ that is not a $q$-th power. Now the polynomial $x^q-\alpha$ is irreducible in $F$, so the ring $E:=F[x]\big/\langle x^q-\alpha\rangle$ is a field extension of $F$ of degree $q$. On the other hand, by the problem hypotheses, $[E:F]$ must also be divisible by $p$, so this forces $q=p$, as desired.