Every point has a fundamental system of neighborhoods given by $\{x+p^n\Bbb Z_p\}_{n\in\Bbb N}$ which are compact.
In essence this is just the fact that $p^n\Bbb Z_p$ is a fundamental system of compact neighborhoods of $0$: since we are in a vector space--really a topological group is enough, but not everyone is familiar with structures of that generality--we can translate these sets anywhere to form a compact (and open) neighborhood of any point.
If you have any trouble seeing this recall that open balls generate the topology and all open balls are of the form $x+p^n\Bbb Z_p$ for some $n\in\Bbb Z$.
We adopt the following notations:
$\mathcal{O}_K$ is the ring of algebraic integers in $K$ and $\mathfrak{p}\subset \mathcal{O}_K$ is a prime of $K$ with $\mathcal{O}_\nu\subset K$ the corresponding valuation ring.
$(p)=\mathfrak{p}\cap\mathbb{Z}$ with $\mathbb{Z}_{(p)}$ the $p$-adic valuation ring of $\mathbb{Q}$. Since $\nu_{\mathfrak{p}}(p)=1$, $p$ is a uniformiser for $\mathcal{O}_\nu$.
The residue field of $K$ is $k_\nu=\mathcal{O}_\nu/\mathfrak{m}_\nu$ where $\mathfrak{m}_\nu=p\mathcal{O}_\nu$.
Write $\mathcal{O}_\mathfrak{p}=\varprojlim \mathcal{O}_\nu/p^n\mathcal{O}_\nu$ for the valuation ring of $K_\mathfrak{p}$ and $k_\mathfrak{p}=\mathcal{O}_\mathfrak{p}/\mathfrak{m}_\mathfrak{p}$ for the residue field with $\mathfrak{m}_\mathfrak{p}$ the maximal ideal in $\mathcal{O}_\mathfrak{p}$.
First we show that $k_\mathfrak{p}\simeq k_\nu$:
Suppose $a:(a_n)\in \mathcal{O}_\mathfrak{p}$ with $|a|=|a_m|\;\forall\; m\ge N$. We then have a homomorphism
$$
\begin{array}{rcl}
\phi\colon \mathcal{O}_\mathfrak{p}&\rightarrow &\mathcal{O}_\nu\\
a&\mapsto&a_N
\end{array}
$$
which is surjective since $\mathcal{O}_\mathfrak{p}$ contains the constant sequences from $\mathcal{O}_\nu$.
We can extend to a surjective map to the quotient:
$$
\phi\colon \mathcal{O}_\mathfrak{p}\rightarrow \mathcal{O}_\nu\rightarrow \mathcal{O}_\nu/ \mathfrak{m}_\nu
$$
with $\ker\phi=\{a\in\mathcal{O}_\mathfrak{p}:|a|<1\}=\mathfrak{m}_\mathfrak{p}$.
Next we show that $k_\nu$ has a subfield $\mathbb{F}_p$:
We have a composite ring homomorphism
$$
\psi\colon\mathbb{Z}\hookrightarrow \mathcal{O}_K\hookrightarrow \mathcal{O}_{K,\mathfrak{p}}=\mathcal{O}_\nu\rightarrow \mathcal{O}_\nu/ p\mathcal{O}_\nu
$$
and $\psi(\mathbb{Z})=\mathbb{Z}/ (p)\simeq\mathbb{F}_p$.
Finally, we show $[k_\nu\colon \mathbb{F}_p]<\infty$:
For $a\in \mathcal{O}_\nu$ write $\bar{a}$ for its image in $k_\nu$. Suppose $\bar{a}_1,\dots,\bar{a}_n$ are linearly independent over $\mathbb{F}_p$. We will show that $a_1,\dots,a_n\in \mathcal{O}_\nu\subset K$ are then linearly independent over $\mathbb{Q}$:
Suppose not and we have
$$
r_1 a_1+\cdots+r_na_n=0,\quad r_i\in\mathbb{Q}.
$$
Suppose wlog that $r_1$ is the non-zero coefficient with minimum $p$-adic valuation. Then if $r_i\ne 0$ we have $\nu_p(r_i/ r_1) \ge 0$. So dividing through by $r_1$, clearing denominators and reducing $\mod p$ we obtain
$$
g_1\bar{a}_1+g_2 \bar{a}_2+\cdots g_n\bar{a}_n=0, \quad g_i\in \mathbb{F}_p, g_1\ne 0.
$$
which would be a contradiction.
So $a_1,\dots,a_n$ are linearly independent over $\mathbb{Q}$ and thus $n\le [K:\mathbb{Q}]$ which is finite.
i.e. $[k_\nu:\mathbb{F}_p]<\infty$ and $k_\mathfrak{p}\simeq k_\nu$ is finite ( and of characteristic $p$).
Best Answer
Proof: take $f\in \Bbb{F}_\ell[x]$, monic irreducible. As $\gcd(p,\ell)=1$ there is $g\in \Bbb{Z}[x]$ monic such that $g\equiv f\bmod \ell, g\equiv x^{\deg(f)}-1\bmod p$.
(if $p$ divides $\deg(f)$ then take instead $g\equiv f\bmod \ell, g\equiv x^{\deg(f)}-x\bmod p$)
By Hensel lemma $g$ has a root in $\Bbb{Q}_p$. This will be a root of $f$ in $O_v/\mathfrak{m}_v$.
Whence $O_v/\mathfrak{m}_v$ contains $\overline{\Bbb{F}}_\ell$.