Let me address your first question. First, I want to argue that there is no precise meaning of "involving numbers only". For example, given a finite field $F$ of size $4$ constructed in the usual manner (quotient of a polynomial ring over $\mathbb{Z}/2\mathbb{Z}$), I can choose a set of numbers, say
$$S=\{37,\tfrac{5}{19},\pi,e\}$$
and, choosing a bijection of $S$ with $F$, use transport of structure to give $S$ the structure of a field. The field structure does not depend in any way on what the underlying set is "made of".
However, along the lines of what I think you are ultimately after, you can obtain finite fields of any possible order using larger rings of integers. For example, $\mathbb{Z}[i]/(3)$ is a finite field of size $9$, and $\mathbb{Z}[i]$ consists of very reasonable numbers,
$$\mathbb{Z}[i]=\{a+bi\mid a,b\in\mathbb{Z}\}.$$
Now let me addressr your second question. Let's use $\mathbb{F}_p$ to mean $\mathbb{Z}/p\mathbb{Z}$, a finite field of order $p$ - it is a very common notation that is slightly less cumbersome, but doesn't mean anything different, they are exact synonyms.
A finite field of order $p^n$ is often constructed by taking the polynomial ring $\mathbb{F}_p[x]$, choosing an irreducible polynomial $f\in \mathbb{F}_p[x]$ of degree $n$, and then making the field
$$F=\mathbb{F}_p[x]/(f).$$
Now, the division algorithm for polynomials tells you that each equivalence class in this quotient can be uniquely identified by a representative of degree $<n$. In other words,
$$\begin{align*}
F&=\{a_0+a_1x+\cdots +a_{n-1}x^{n-1}+(f)\mid a_0,a_1,\ldots,a_{n-1}\in\mathbb{F}_p\}\\\\
&=\left\{\,\overline{a_0+a_1x+\cdots +a_{n-1}x^{n-1}}\,\;\middle\vert\;a_0,a_1,\ldots,a_{n-1}\in\mathbb{F}_p\right\}\\\\\\
&=\{a_0+a_1\overline{x}+\cdots +a_{n-1}\overline{x}^{n-1}\mid a_0,a_1,\ldots,a_{n-1}\in\mathbb{F}_p\}
\end{align*}$$
Letting the symbol $\alpha$ be a stand-in for $\overline{x}$, you can think of $F$ as being $\mathbb{F}_p$ with a new element "$\alpha$" added in, where $\alpha$ is a root of $f$, and you can write $F=\mathbb{F}_p[\alpha]$.
Now, the prime subfield of $F$ is just the "constant" polynomials, i.e. the ones with no $\alpha$'s in them:
$$\text{the prime subfield of }F=\{a_0+0\alpha+\cdots+0\alpha^{n-1}\mid a_0\in\mathbb{F}_p\}$$
and for each divisor $d\mid n$, the unique subfield of $F$ of order $p^d$ is the collection of polynomials in $\alpha$ whose terms are those of exponents that are multiples of $n/d$:
$$\text{the subfield of }F\text{ of order }p^d=\{a_0+a_1\alpha^{n/d}+\cdots+a_{d-1}\alpha^{(d-1)n/d}\mid a_0,a_1,\ldots,a_{d-1}\in\mathbb{F}_p\}$$
(clearly, the above set has cardinality $p^d$, because it takes $d$ elements of $\mathbb{F}_p$ to specify a given element of the above set, namely, each of the coefficients of the powers of $\alpha$. To see that it is a field, remember that $(a+b)^p=a^p+b^p$ in a field of characteristic $p$.)
In $\mathbb Z_p$ and hence its extension $F$ we have
$$
p a = \underbrace{a + \cdots + a}_{p \text{ times}} = (\underbrace{1 + \cdots + 1}_{p \text{ times}})a = 0 \cdot a = 0.
$$
By the binomial theorem
$$
(a + b)^{p^n} = \sum_{k=0}^{p^n}\binom{p^n}{k} a^k b^{p^n - k}.
$$
All terms in the latter expansion are multiples of $p$ except for $a^{p^n}$ and $b^{p^n}$. It follows that in $\mathbb Z_p$ and $F$ we have
$$(a + b)^{p^n} = a^{p^n} + b^{p^n}.$$
Now show that the roots of $f(x)$ are closed under addition, multiplication and taking additive and multiplicative inverses. This shows that $F$ consists entirely of the roots of $f(x)$. We conclude that the order of $F$ is $p^n$ as desired.
Best Answer
Hint: Such a field is the splitting field of $X^{p^n}-X=0$ and contains the splitting field of $X^{p^m}-X$ if $m$ divides $n$.