You seem to mean two slightly different things by the $p=\ell$ case. Your first theorem is about a field of characteristic $p$, and the second is about $\mathbb C_p$, which is a field of characteristic zero. Let's talk first about the significance of this.
For a field of actual characteristic $p$, the proof of "Artin vanishing" for mod $p$ coefficients is even easier than you gave. One doesn't need any perfectoid spaces, as you can just use the Artin-Schreier sequence $0 \to \mathbb F_p \to \mathcal O_X \to \mathcal O_X \to 0$ to handle the case of affine schemes.
By a Lefschetz principle argument, your proof over $\mathbb C_p$ can be adapted to work over an arbitrary field of characteristic zero. Of course, by the same argument, and the comparison theorem between etale and analytic cohomology, the Artin affine theorem can also be proved by analytic arguments over fields of characteristic zero (say, using a Morse function).
Characteristic zero really is pretty much always easier than characteristic $p$.
In actual characteristic $p$, the $\ell=p$ case is not necessarily always easier. I think a good description is that it is more degenerate. The cohomology groups are usually much smaller, in fewer degrees. It's not surprising that vanishing results are easier to prove in this setting! But other results we want, like Poincare duality or Euler characteristic formulas, are simply not true for mod $p$ coefficients.
There is certainly something unsatisfying about these somewhat ad hoc arguments, but they're not THAT bad. I think getting over some of your discomfort with them might be easier than finding a better argument.
It's perhaps worthwhile to point out that Grothendieck famously wanted to solve problems by setting up theories that make the problems easy rather than the concrete calculations favored by mathematicians like Serre and Deligne, which were later necessary to solve problems like the Weil conjectures. If you find the arguments in the part of etale cohomology worked out by Grothendieck to have too many concrete calculations and not enough big theories then I am tempted to say you have gone too far, but I perhaps shouldn't because many fields of mathematics have trended in that direction since Grothendieck's day, and that perspective does work for at least some mathematicians.
Anyways, the use of $\mathbb P^1$ in this argument is certainly not a pure application of general theory, but it is also not ad hoc. It is part of a systematic strategy in the foundations of etale cohomology, which is to first prove the desired statement for curves using the powerful theorems available for curves (precise descriptions of $H^1$ and $H^2$, together with dimension estimates), and then find an inductive approach to the general case based on varieties mapping to a curve with fibers mapping to a curve, etc., suitable to the particular problem.
For different problems, it makes sense to try slightly different curves, and put them together in slightly different ways.
For the affine vanishing theorem, we of course want to look at affine curves, so we might look first at the simplest such, $\mathbb A^1$, and then observe we can prove the theorem for arbitrary products of $\mathbb A^1$, or $\mathbb A^n$. Can we reduce from the general case to $\mathbb A^n$? Yes, by Noether normalization.
For the comparison of etale and analytic cohomology, it's convenient to work with affine curves, and fibrations of affine curves by affine curves, so that all our cohomology will match the cohomology of the fundamental group, as then we can take advantage of the comparison of the etale and analytic fundamental groups. In this case, it suffices to check that smooth varieties are covered by open subsets isomorphic to an iterated fibration of curves, and this we can do.
For the proper base change theorem, we of course want to start with proper curves, and so we try the simplest such, $\mathbb P^1$. There's no clear reason to take nontrivial fibrations of $\mathbb P^1$ over $\mathbb P^1$, so we just look at powers, $(\mathbb P^1)^n$. Can we reduce from a general projective variety to $(\mathbb P^1)^n$? Not directly, as we only know that projective varieties map to $\mathbb P^n$, but using the map $(\mathbb P^1)^n \to \mathbb P^n$ we can close the gap.
So in some sense it's all the same idea, just adapated in different ways to different contexts.
The Weil conjectures are one place this strategy failed, as Grothendieck tried to prove them by showing every smooth projective variety is covered by a product of curves, but this is false. (Deligne was later, in Weil II, able to solve the problem by an inductive argument, using the theory of weights of constructible sheaves.)
Best Answer
The main case can be found in Milne's article specifically Theorems 1.13, 1.14 on page 310. The idea, briefly, is as follows: Given a sheaf $F$ on a variety $X$ over a finite field $k$, then over an algebraic closure $\bar{k}$ of $k$, the group $H^i_{et}(\bar{X}, F)$ becomes a $Gal(\bar{k}/k)$-module. There is a spectral sequence involving the $H^j(Gal(\bar{k}/k), H^i_{et}(\bar{X}, F))$ which converges to $H^n_{et}(X,F)$. This is true over any perfect field.
When you have duality over $\bar{k}$ (e.g. $X$ smooth proper and $F$ nice), combine it with duality in Galois cohomology (in our case, the group is very simple: $\hat{Z}$) to get duality over $k$. The duality theorems now reflect the $k$: if Poincare duality for $X$ of dimension $d$ over $\bar{k}$ pairs $H^i$ with $H^{2d-i}$, over $k$ the pairing will be between $H^i$ and $H^{2d +m -i}$ where $m$ is the cohomological dimension (assumed finite) of the Galois group ($m=1$ in the case of a finite field).
Hope this helps.