There are very good experimental limits on light neutrinos that have the same electroweak couplings as the neutrinos in the first 3 generations from the measured width of the $Z$ boson. Here light means $m_\nu < m_Z/2$. Note this does not involve direct detection of neutrinos, it is an indirect measurement based on the calculation of the $Z$ width given the number of light neutrinos. Here's the PDG citation:
http://pdg.lbl.gov/2010/listings/rpp2010-list-number-neutrino-types.pdf
There is also a cosmological bound on the number of neutrino generations coming from production of Helium during big-bang nucleosynthesis. This is discussed in "The Early Universe" by Kolb and Turner although I am sure there are now more up to date reviews. This bound is around 3 or 4.
There is no direct relationship between quark and neutrino masses, although you can derive
possible relations by embedding the Standard Model in various GUTS such as those based on
$SO(10)$ or $E_6$. The most straightforward explanation in such models of why neutrinos are light is called the see-saw mechanism
http://en.wikipedia.org/wiki/Seesaw_mechanism
and leads to neutrinos masses $m_\nu \sim m_q^2/M$ where $M$ is some large mass scale on the order of $10^{11} ~GeV$ associated with the vacuum expectation value of some Higgs field that plays a role in breaking the GUT symmetry down to $SU(3) \times SU(2) \times U(1)$. If the same mechanism is at play for additional generations one would expect the neutrinos to be lighter than $M_Z$ even if the quarks are quite heavy.
Also, as you mentioned, if you try to make fourth or higher generations very heavy you have to increase the Yukawa coupling to the point that you are outside the range of perturbation theory. These are rough theoretical explanations and the full story is much more complicated but the combination of the excellent experimental limits, cosmological bounds and theoretical expectations makes most people skeptical of further generations. Sorry this wasn't mathier.
The question: "I'm wondering if there is some good reason why the universe as we know it has to have twelve particles rather than just four."
The short answer: Our current standard description of the spin-1/2 property of the elementary particles is incomplete. A more complete theory would require that these particles arrive in 3 generations.
The medium answer: The spin-1/2 of the elementary fermions is an emergent property. The more fundamental spin property acts like position in that the Heisenberg uncertainty principle applies to consecutive measurements of the fundamental spin the same way the HUP applies to position measurements. This fundamental spin is invisible to us because it is renormalized away. What's left is three generations of the particle, each with the usual spin-1/2.
When a particle moves through positions it does so by way of an interaction between position and momentum. These are complementary variables. The equivalent concept for spin-1/2 is "Mutually unbiased bases" or MUBs. There are only (at most) three MUBs for spin-1/2. Letting a particle's spin move among them means that the number of degrees of freedom of the particle have tripled. So when you find the long time propagators over that Hopf algebra you end up with three times the usual number of particles. Hence there are three generations.
The long answer: The two (more or less classical) things we can theoretically measure for a spin-1/2 particle are its position and its spin. If we measure its spin, the spin is then forced into an eigenstate of spin so that measuring it again gives the same result. That is, a measurement of spin causes the spin to be determined. On the other hand, if we measure its position, then by the Heisenberg uncertainty principle, we will cause an unknown change to its momentum. The change in momentum makes it impossible for us to predict the result of a subsequent position measurement.
As quantum physicists, we long ago grew accustomed to this bizarre behavior. But imagine that nature is parsimonious with her underlying machinery. If so, we'd expect the fundamental (i.e. before renormalization) measurements of a spin-1/2 particle's position and spin to be similar. For such a theory to work, one must show that after renormalization, one obtains the usual spin-1/2.
A possible solution to this conundrum is given in the paper:
Found.Phys.40:1681-1699,(2010), Carl Brannen, Spin Path Integrals and Generations
http://arxiv.org/abs/1006.3114
The paper is a straightforward QFT resummation calculation. It assumes a strange (to us) spin-1/2 where measurements act like the not so strange position measurements. It resums the propagators for the theory and finds that the strange behavior disappears over long times. The long time propagators are equivalent to the usual spin-1/2. Furthermore, they appear in three generations. And it shows that the long time propagators have a form that matches the mysterious lepton mass formulas of Yoshio Koide.
Peer review: The paper was peer-reviewed through an arduous process of three reviewers. As with any journal article it had a managing editor, and a chief editor. Complaints about the physics have already been made by competent physicists who took the trouble of carefully reading the paper. It's unlikely that someone making a quick read of the paper is going to find something that hasn't already been argued through. The paper was selected by the chief editor of Found. Phys. as suitable for publication in that journal and so published last year.
The chief editor of Found. Phys. is now Gerard 't Hooft. His attitude on publishing junk is quite clear, he writes
How to become a bad theoretical physicist
On your way towards becoming a bad
theoretician, take your own immature
theory, stop checking it for mistakes,
don't listen to colleagues who do spot
weaknesses, and start admiring your
own infallible intelligence. Try to
overshout all your critics, and have
your work published anyway. If the
well-established science media refuse
to publish your work, start your own
publishing company and edit your own
books. If you are really clever you
can find yourself a formerly
professional physics journal where the
chief editor is asleep.
http://www.phys.uu.nl/~thooft/theoristbad.html
One hopes that 't Hooft wasn't asleep when he allowed this paper to be published.
Extensions: My next paper on the subject extends the above calculation to obtain the weak hypercharge and weak isospin quantum numbers. It uses methods similar to the above, that is, the calculation of long time propagators, but uses a more sophisticated method of manipulating the Feynman diagrams called "Hopf algebra" or "quantum algebra". I'm figuring on sending it in to the same journal. It's close to getting finished, I basically need to reread it over and over and add references:
http://brannenworks.com/E8/HopfWeakQNs.pdf
Best Answer
We don't have a good explanation for why the quarks and leptons fall into generations. But we have some very strong arguments that it has to be this way, because of the way the weak interactions behave.
First, the weak interactions tell us that each lepton should be paired with a neutrino, and that each charge 2/3 quark should be paired with a charge -1/3 quark. This pairing is necessary just to write down the Lagrangian for the weak interactions.
The second bit is even weirder. The weak interactions are chiral; they don't treat left-handed particles in the same way that they treat right-handed particles. Quantum chiral gauge theories, like the SU(2) x U(1) gauge theory describing the electroweak interactions, are somewhat delicate beasts. Most classical chiral gauge theories can not be quantized; quantum mechanical effects give rise to anomalous gauge symmetry breaking, which ruin the consistency of the theory.
In the case of the Glashow-Weinberg-Salam model, there's a consistency condition for avoiding anomalies: 3 times the sum of the charges in a quark doublet + the sum of the charges in a lepton doublet must equal zero. This condition is satisfied by the Standard Model particles: 3(2/3 - 1/3) + (0 - 1) = 0. Which tells us that the quark and lepton doublets in a generation really are paired in a non-trivial way. If they weren't paired up, the theory would most likely be inconsistent.