First of all, I am aware of the questions about the Zariski topology asked here and I am also aware of the discussion at the Secret Blogging Seminar. But I could not find an answer to a question that bugged me right from my first steps in algebraic geometry: how can I really motivate the Zariski topology on a scheme?
For example in classical algebraic geometry over an algebraically closed field I can define the Zariski topology as the coarsest $T_1$-topology such that all polynomial functions are continuous. I think that this is a great definition when I say that I am working with polynomials and want to make my algebraic set into a local ringed space. But what can I say in the general case of an affine scheme?
Of course I can say that I want to have a fully faithful functor from rings into local ringed spaces and this construction works, but this is not a motivation.
For example for the prime spectrum itself, all motivations I came across so far are as follows: well, over an algebraically closed field we can identify the points with maximal ideals, but in general inverse images of maximal ideals are not maximal ideals, so let's just take prime ideals and…wow, it works. But now that I know that one gets the prime spectrum from the corresponding functor (one can of course also start with a functor) by imposing an equivalence relation on geometric points (which I find very geometric!), I finally found a great motivation for this. What is left is the Zariski topology, and so far I just came across similar strange motivations as above…
Best Answer
In the category of sets there is no such thing as the initial local ring into which R maps, i.e. a local ring L and a map f:R-->L such that any map from R into a local ring factors through f.
But a ring R is a ring object in the topos of Sets. Now if you are willing to let the topos vary in which it should live, such a "free local ring on R" does exist: It is the ring object in the topos of sheaves on Spec(R) which is given by the structure sheaf of Spec(R). So the space you were wondering about is part of the solution of forming a free local ring over a given ring (you can reconstruct the space from the sheaf topos, so you could really say that it "is" the space).
Edit: I rephrase that less sloppily, in response to Lars' comment. So the universal property is about pairs (Topos, ring object in it). A map (T,R)-->(T',R') between such is a pair
(adjunction $f_*:T \leftrightarrow T':f^* $ , morphism of ring objects $f^*R'\rightarrow R$).
Note that by convention the direction of the map is the geometric direction, the one corresponding to the direction of a map topological spaces - in my "universal local ring" picture I was stressing the algebraic direction, which is given by $f^*$.
Now for a ring R there is a map $(Sh(Spec(R)), O_{Spec(R)})\rightarrow(Set,R)$: $f^* R$ is the constant sheaf with value R on Spec(R), the map $f^* R \rightarrow O_{Spec(R)}$ is given by the inclusion of R into its localisations which occur in the definition of $O_{Spec(R)}$. This is the terminal map (T,L)-->(Set,R) from pairs with L a local ring. For a simple example you might want to work out how such a map factors, if the domain pair happens to be of the form (Set,L).
This universal property of course determines the pair up to equivalence. It thus also determines the topos half of the pair up to equivalence, and thus also the space Spec(R) up to homeomorphism.(end of edit)
An even nicer reformulation of this is the following (even more high brow, but to me it gives the true and most illuminating justification for the Zariski topology, since it singles out just the space Spec(R)):
A ring R, i.e. a ring in the topos of sets, is the same as a topos morphism from the topos of sets into the classifying topos T of rings (by definition of classifying topos). There also is a classifying topos of local rings with a map to T (which is given by forgetting that the universal local ring is local). If you form the pullback (in an appropriate topos sense) of these two maps you get the topos of sheaves on Spec(R) (i.e. morally the space Spec(R)). The map from this into the classifying topos of local rings is what corresponds to the structure sheaf.
Isn't that nice? See Monique Hakim's "Schemas relatifs et Topos anelles" for all this (the original reference, free of logic), or alternatively Moerdijk/MacLane's "Sheaves in Geometry and Logic" (with logic and formal languages).