Hi Anweshi,
Since Emerton answered your third grey-boxed question very nicely, let me try at the first two. Suppose $L(s,f)$ is one of the L-functions that you listed (including the first two, which we might as well call L-functions too). (For simplicity we always normalize so the functional equation is induced by $s\to 1-s$.) This guy has an expansion $L(s,f)=\sum_{n}a_f(n)n^{-s}$ as a Dirichlet series, and the most general prime number theorem reads
$\sum_{p\leq X}a_f(p)=r_f \mathrm{Li}(x)+O(x \exp(-(\log{x})^{\frac{1}{2}-\varepsilon})$.
Here $\mathrm{Li}(x)$ is the logarithmic integral, $r_f$ is the order of the pole of $L(s,f)$ at the point $s=1$, and the implied constant depends on $f$ and $\varepsilon$.
Let's unwind this for your examples.
1) The Riemann zeta function has a simple pole at $s=1$ and $a_f(p)=1$ for all $p$, so this is the classical prime number theorem.
2) The Dedekind zeta function (say of a degree d extension $K/\mathbb{Q}$) is a little different. It also has a simple pole at $s=1$, but the coefficients are determined by the rule: $a(p)=d$ if $p$ splits completely in $\mathcal{O}_K$, and $a(p)=0$ otherwise. Hence the prime number theorem in this case reads
$|p\leq X \; \mathrm{with}\;p\;\mathrm{totally\;split\;in}\;\mathcal{O}_K|=d^{-1}\mathrm{Li}(x)+O(x \exp(-(\log{x})^{\frac{1}{2}-\varepsilon})$.
This already has very interesting applications: the fact that the proportion of primes splitting totally is $1/d$ was very important in the first proofs of the main general results of class field theory.
3) If $\rho:\mathcal{G}_{\mathbb{Q}}\to \mathrm{GL}_n(\mathbb{C})$ is an Artin representation then $a(p)=\mathrm{tr}\rho(\mathrm{Fr}_p)$. If $\rho$ does not contain the trivial representation, then $L(s,\rho)$ has no pole in neighborhood of the line $\mathrm{Re}(s)\geq 1$, so we get
$\sum_{p\leq X}\mathrm{tr}\rho(\mathrm{Fr}_p)=O(x \exp(-(\log{x})^{\frac{1}{2}-\varepsilon})$.
The absence of a pole is not a problem: it just means there's no main term! In this particular case, you could interpret the above equation as saying that "$\mathrm{tr}\rho(\mathrm{Fr}_p)$ has mean value zero.
4) For an elliptic curve, the same phenomenon occurs. Here again there is no pole, and $a(p)=\frac{p+1-|E(\mathbb{F}_p)|}{\sqrt{p}}$. By a theorem of Hasse these numbers satisfy $|a(p)|\leq 2$, so you could think of them as the (scaled) deviation of $|E(\mathbb{F}_p)|$ from its
"expected value" of $p+1$. In this case the prime number theorem reads
$\sum_{p\leq X}a(p)=O(x \exp(-(\log{x})^{\frac{1}{2}-\varepsilon})$
so you could say that "the average deviation of $|E(\mathbb{F}_p)|$ from $p+1$ is zero."
Now, how do you prove generalizations of the prime number theorem? There are two main steps in this, one of which is easily lifted from the case of the Riemann zeta function.
Prove that the prime number theorem for $L(s,f)$ is a consequence of the nonvanishing of $L(s,f)$ in a region of the form $s=\sigma+it,\;\sigma \geq 1-\psi(t)$ with $\psi(t)$ positive and tending to zero as $t\to \infty$. So this is some region which is a very slight widening of $\mathrm{Re}(s)>1$. The proof of this step is essentially contour integration and goes exactly as in the case of the $\zeta$-function.
Actually produce a zero-free region of the type I just described. The key to this is the existence of an auxiliary L-function (or product thereof) which has positive coefficients in its Dirichlet series. In the case of the Riemann zeta function, Hadamard worked with the auxiliary function $ A(s)=\zeta(s)^3\zeta(s+it)^2 \zeta(s-it)^2 \zeta(s+2it) \zeta(s-2it)$. Note the pole of order $3$ at $s=1$; on the other hand, if $\zeta(\sigma+it)$ vanished then $A(s)$ would vanish at $s=\sigma$ to order $4$. The inequality $3<4$ of order-of-polarity/nearby-order-of-vanishing leads via some analysis to the absence of any zero in the range $s=\sigma+it,\;\sigma \geq 1-\frac{c}{\log(|t|+3)}.$ In the general case the construction of the relevant auxiliary functions is more complicated. For the case of an Artin representation, for example, you can take $B(s)=\zeta(s)^3 L(s+it,\rho)^2 L(s-it,\widetilde{\rho})^2 L(s,\rho \otimes \widetilde {\rho})^2 L(s+2it,\rho \times \rho) L(s-2it,\widetilde{\rho} \times \widetilde{\rho})$. The general key is the Rankin-Selberg L-functions, or more complicated L-functions whose analytic properties can be controlled by known instances of Langlands functoriality.
If you'd like to see everything I just said carried out elegantly and in crystalline detail, I can do no better than to recommend Chapter 5 of Iwaniec and Kowalski's book "Analytic Number Theory."
I like the proof by Paul Monsky:
'Simplifying the Proof of Dirichlet's Theorem'
American Mathematical Monthly, Vol. 100 (1993), pp. 861-862.
Naturally this does maintain the distinction between real and complex
as whatever you do, the complex case always seems to be easier
as one would have two vanishing L-functions for the price of one.
I incorporated this argument into my note on a "real-variable" proof
of Dirichlet's theorem at
http://secamlocal.ex.ac.uk/people/staff/rjchapma/etc/dirichlet.pdf .
There are proofs, notably in Serre's Course in Arithmetic
which claim to treat the real and complex case on the same footing.
But this is an illusion; it pretends the complex case is as hard
as the real case. Serre considers the product $\zeta_m(s)=\prod L(s,\chi)$
where $\chi$ ranges over the modulo $m$ Dirichlet characters.
If one of the $L(1,\chi)$ vanishes then $\zeta_m(s)$ is bounded as $s\to 1$
and Serre obtains a contradiction by using Landau's theorem on
the abscissa of convergence of a positive Dirichlet series. But all
this subtlety is only needed for the case of real $\chi$. In the non-real
case, at least two of the $L(1,\chi)$ vanish so that $\zeta_m(s)\to0$
as $s\to1$. But it's elementary that $\zeta_m(s)>1$ for real $s>1$
and the contradiction is immediate, without the need of Landau's subtle result.
Added (25/5/2010) I like the Ingham/Bateman method. It is superficially
elegant, but as I said in the comments, it makes the complex case as hard
as the real. Again it reduces to using Landau's result or a choice of other
trickery.
What one should look at is not $\zeta(s)^2L(s,\chi)L(s,\overline\chi)$
but
$$G(s)=\zeta(s)^6 L(s,\chi)^4 L(s,\overline\chi)^4 L(s,\chi^2)L(s,\overline\chi^2)$$
(cf the famous proof of nonvanishing of $\zeta$ on $s=1+it$ by
Mertens).
Unless $\chi$ is real-valued this function will vanish at $s=1$ if
$L(1,\chi)=0$. But one shows that $\log G(s)$ is a Dirichlet
series with nonnegative coefficients and we get an immediate contradiction
without any subtle lemmas. Again it shows that the real case is the hard one.
For real $\chi$ then $G(s)=[\zeta(s)L(s,\chi)]^8$ while Ingham/Bateman
would have us consider $[\zeta(s)L(s,\chi)]^2$. This leads us to
the realization that for real $\chi$ we should look at $\zeta(s)L(s,\chi)$
which is the Dedekind zeta function of a quadratic field. (So if one
is minded to prove the nonvanishing by showing that a Dedekind zeta function
has a pole, quadratic fields suffice, and one needn't bother with cyclotomic
fields).
But we can do more. Let $t$ be real and consider
$$G_t(s)=
\zeta(s)^6 L(s+it,\chi)^4 L(s-it,\overline\chi)^4
L(s+2it,\chi^2)L(s-2it,\overline\chi^2).$$
Unless both $t=0$ and $\chi$ is real, if $L(1+it,\chi)=0$ one gets
a contradiction just as before. So the nonvanishing of any $L(s,\chi)$
on the line $1+it$ is easy except at $1$ for real $\chi$.
This special case really does seem to be deeper!
Added (26/5/2010)
The argument I outlined with the function $G_t(s)$ is well-known to extend
to a proof for a zero-free region of the L-function to the left of the line
$1+it$. At least it does when unless $t=0$ and $\chi$ is real-valued.
In that case it breaks down and we get the phenomenon of the Siegel zero;
the possible zero of $L(s,\chi)$ for $\chi$ real-valued, just to the left of
$1$ on the real line. So the extra difficulty of proving $L(1,\chi)\ne0$
for $\chi$ real-valued is liked to the persistent intractability of
showing that Siegel zeroes never exist.
Best Answer
A MathSciNet search set to Books and with "arithmetic functions" entered into the "Anywhere" field yields 148 matches. Some of the more promising ones:
$ $
$ $
$ $
$ $
$ $
$ $