I'd like to know if anyone has a good explanation for where the ghost components that are used to define Witt vectors come from. A lot of sources I've read take the ghost components for their starting point.
The closest I've seen to an explanation was in Joe Rabinoff's notes on Witt vectors. This is my paraphrasing what I understood from them. Consider $p$-typical Witt vectors and let $k$ be a perfect field (or ring) of characteristic $p$. We have a multiplicative Teichmüller map $[\ ]: k \rightarrow W(k)$ which is a section to the projection map. Every element of $W(k)$ can be uniquely written as a series $$\sum_{i=0}^{\infty} p^i[a_i],$$ with $a_i \in k$. If we know how to add witt vectors, then we will know how to multiply them as well, because the Teichmüller map is multiplicative.
The series expression for $[a_0] + [b_0]$ is already not obvious. Write $\sum p^i [c_i]$ for this expression. We have $c_0 = a_0 + b_0$, but to find $c_1$ we need the following trick. Because our field is perfect, there is no harm in replacing $a_0$ and $b_0$ by $\alpha^p$ and $\beta^p$. Then we want to compute $[\alpha^p] + [\beta^p] = [\alpha]^p + [\beta]^p.$ We know $[\alpha] + [\beta] \equiv [\alpha + \beta] \mod p$. Raising both sides to the $p$th power, we can find a new expression for $[\alpha]^p + [\beta]^p \mod p^2$, and this gives us $c_1$.
The formulas that are used to find $c_1$, $c_2$, etc. look similar to the ghost polynomials. [They're a little different because the Witt vector corresponding to a series as above is $(c_0, c_1^p, c_2^{p^2}, \ldots)$, and so we haven't found the usual Witt vector components. On the other hand, the usual ghost components don't involve any $p$th roots, which is what $\alpha$ and $\beta$ are. Finally, the ghost map tells us not just how to add $[a_0]$ and $[b_0]$, but how to add any two series $\sum p^i[a_i]$ and $\sum p^i[b_i]$.]
I'm willing to believe that we can find precisely the usual ghost components this way (although I haven't actually succeeded in doing this). So it seems natural that the ghost map $w: W(k) \rightarrow k^{\mathbb{N}}$ (where the ring operations on $k^{\mathbb{N}}$ are componentwise) is an additive map. What I find surprising is that it's also multiplicative; it seems like we didn't do anything to guarantee this.
Am I correct that this is one way to find the ghost components? Is this the most natural path to them? Should it be obvious that the ghost map is multiplicative?
I'd be happy to hear how you think of the ghost components.
Best Answer
I hope this question ist still interesting for some people, as it is for me. I will first try to give a motivation for the "ghost map", i.e. the Witt polynomials, which, I think, is absent from Harder's article, and then give some historical remarks.
Motivation (partly inspired by the impressive article on Witt vectors in the German wikipedia): For this forget for a moment that we know the Teichmüller representatives. Imagine we have a strict $p$-ring $A$ with (perfect) residue ring $k$, that is, for any set-theoretic section $\sigma: k \rightarrow A$ of $A \twoheadrightarrow k$, every element of A can be written as
$$\sum_{i=0}^\infty \sigma(a_i) p^i$$
with unique $a_i$ in $k$. That's a set-theoretic bijection
$$k^\mathbb{N} \leftrightarrow A .$$
How to describe the ring structure on the "coordinates" on the left? It suffices to describe it mod $p^{n+1}$ for every n. The most naive "coordinate map" would be
$$k^{n+1} \rightarrow A/p^{n+1} A$$
$$(a_0, ..., a_n) \mapsto \sigma(a_0) + \sigma(a_1) p + ... + \sigma(a_n)p^n .$$
To this, there would correspond the naiveWitt polynomial
$$NW(X_0, ..., X_{n}) = X_0 + pX_1 + ... + p^n X_n .$$
"Naive" because the above map induced by it depends on $\sigma$. (We could even do worse and choose different $\sigma$'s for each coordinate, still maintaining a bijection). Now do remember -- not the Teichmüller representatives, but the crucial fact from their construction: $a \equiv b$ mod $p \Rightarrow a^{p^i} \equiv b^{p^i}$ mod $p^{i+1}$. All possible $\sigma(a_0)$ are congruent mod $p$; we want something unique mod $p^{n+1}$, so why not write $\sigma(a_0)^{p^n}$ in the 0-th coordinate. The first coordinate will be multiplied by $p$ anyway, so we only have to raise it to the $p^{n-1}$-th power to make it unique mod $p^{n+1}$. Upshot:
$$\sigma(a_0)^{p^n} + \sigma(a_1)^{p^{n-1}} p + ... + \sigma(a_n) p^n \in A/p^{n+1} A$$
is independent of $\sigma$; in a way, it is a canonical representative in $A/p^{n+1} A$ of one element in $k^{n+1}$. And it is induced by the (non-naive) Witt polynomial
$$W(X_0, ..., X_n) = X_0^{p^n} + p X_1^{p^{n-1}} + ... + p^n X_n .$$
In other words: For fixed $(a_0, ..., a_n)$, whatever liftings $\sigma_i$ one might choose, evaluating the Witt polynomial at $X_i = \sigma_i(a_i)$ will give the same element in $A/p^{n+1} A$. So it looks like it might produce universal formulae for a ring structure. (Still, it is astounding that these turn out to be polynomials.) Finally, if $k$ is perfect, this coordinate map is still bijective, and we can and will normalise it. Depending on whether one considers the normalisation
$(a_0, ..., a_n) \mapsto (a_0^{p^{-n}}, ..., a_n^{p^{-n}})$ or
$(a_0, ..., a_n) \mapsto (a_0^{p^{-n}}, a_1^{p^{1-n}}, ..., a_n)$
to be more natural, in the limit one will look at the element
$\sum_{i=0}^\infty \tau (a_i^{p^{-i}}) p^i$ or
$\sum_{i=0}^\infty \tau (a_i) p^i$
as a natural representative in $A$ of the coordinates $(a_0, a_1, ...)$ -- where the map $\tau: a \mapsto \lim \sigma (a^{p^{-i}})^{p^i}$ is independent of $\sigma$ and turns out to be the Teichmüller map, which we have thus generalised by forgetting it for a while.
History: The relevant papers are
(Beware, obsolete notation: "perfekt" $\sim$ complete; "(un)vollkommen" = (im)perfect)
In (1), a structure theory of complete discretely valued fields had already been done (!), although more complicated. As olli_jvn has already said, Witt was mainly working on generalising Artin-Schreier theory to what is now Artin-Schreier-Witt theory as well as constructing cyclic algebras of degree $p^n$. This is also what Schmid did in (2). This paper was discussed in an Arbeitsgemeinschaft led by Witt with participants Hasse, Teichmüller, Schmid and others. On the first page of (2), there is a note added during correction that Witt has found a "neues Kalkül" which simplifies Schmid's results. -- Hazewinkel notes (p. 5) that the Witt polynomials turn up in (3) and suggests that this might have inspired Witt, however if one looks where they come from here, one reads (p. 155 = p. 57 in Teichmüller's Collected Works):
and then the Witt polynomials appear, and the summation polynomials (at least, mod p) are deduced from them. On the first page of (3), Teichmüller writes that this work was inspired by the mentioned Arbeitsgemeinschaft. In the introductions to (4) and (5), Witt and Teichmüller credit each other with realising the use of the "neues Kalkül" for the structure theory of complete discretely valued fields in the unequal characteristic case. As Hazewinkel writes (p. 9, in accordance with Witt's introduction in (4)), a decisive inspiration for Witt had been the "summation" polynomials that occured in (2), which are constructed recursively (in building an algebra of degree $p^n$ recursively by adding Artin-Schreier-like $p$-layers), and which for $n = 1$ reduce to a plain sum. Indeed, on p. 111 of (2), there are polynomials $z_\nu$ which in today's notation would be Witt's $S_\nu - X_\nu - Y_\nu$, defined recursively with the help of a polynomial $f_\nu$ to be found on p. 112, which is nothing else than the Witt polynomial $W_\nu$ in slightly different normalization. So presumably the timeline is:
Schmid presents his paper in the Arbeitsgemeinschaft (before January 1936) $\rightarrow$ Witt finds general Witt vector "Kalkül" (January 1936) $\rightarrow$ Witt and Teichmüller independently realise that this gives a structure theory of complete discretely valued fields with perfect residue field; Teichmüller finds sum and product polynomials (mod p) as well as Teichmüller representatives and reduction of the general case to the case of perfect residue field (January-February 1936) $\rightarrow$ Witt works out his whole theory, Witt and Teichmüller agree to put the perfect case among all the other applications into (4), a detailed treatment of the imperfect case in (5).