A Grothendieck universe is known in set theory as the set Vκ for a (strongly) inaccessible cardinal κ. They are exactly the same thing. Thus, the existence of a Grothendieck universe is exactly equivalent to the existence of one inaccessible cardinal. These cardinals and the corresponding universes have been studied in set theory for over a century.
The Grothendieck Universe axiom (AU) is the assertion that every set is an element of a universe in this sense. Thus, it is equivalent to the assertion that the inaccessible cardinals are unbounded in the cardinals. In other words, that there is a proper class of inaccessible cardinals. This is the axiom you sought, which is exactly equivalent to AU. In this sense, the axiom AU is a statement in set theory, having nothing necessarily to do with category theory.
The large cardinal axioms are fruitfully measured in strength not just by direct implication, but also by their consistency strength. One large cardinal property LC1 is stronger than another LC2 in consistency strength if the consistency of ZFC with an LC1 large cardinal implies the consistency of ZFC with an LC2 large cardinal.
Measured in this way, the AU axiom has a stronger consistency strength than the existence of any finite or accessible number of inaccessible cardinals, and so one might think it rather strong. But actually, it is much weaker than the existence of a single Mahlo cardinal, the traditional next-step-up in the large cardinal hierarchy. The reason is that if κ is Mahlo, then κ is a limit of inaccessible cardinals, and so Vκ will satisfy ZFC plus the AU axiom. The difference between AU and Mahloness has to do with the thickness of the class of inaccessible cardinals. For example, strictly stronger than AU and weaker than a Mahlo cardinal is the assertion that the inaccessible cardinals form a stationary proper class, an assertion known as the Levy Scheme (which is provably equivconsistent with some other interesting axioms of set theory, such as the boldface Maximality Principle, which I have studied a lot). Even Mahlo cardinals are regarded as rather low in the large cardinal hierarchy, far below the weakly compact cardinals, Ramsey cardinals, measurable cardinals, strong cardinals and supercompact cardinals. In particular, if δ is any of these large cardinals, then δ is a limit of Mahlo cardinals, and certainly a limit of strongly inaccessible cardinals. So in particular, Vδ will be a model of the AU axiom.
Rather few of the large cardinal axioms imnply AU directly, since most of them remain true if one were to cut off the universe at a given inaccessible cardinal, a process that kills AU. Nevertheless, implicit beteween levels of the large caridnal hiearchy are the axioms of the same form as AU, which assert an unbounded class of the given cardinal. For example, one might want to have unboundedly many Mahlo cardinals, or unboundedly many measurable cardinals, and so on. And the consistency strength of these axioms is still below the consistency strength of a single supercompact cardinal. The hierarchy is extremely fine and intensely studied. For example, the assertion that there are unboundedly many strong cardinals is equiconsistent with the impossiblity to affect projective truth by forcing. The existence of a proper class of Woodin cardinals is particularly robust set-theoretically, and all of these axioms are far stronger than AU.
There are natural weakenings of AU that still allow for almost all if not all of what category theorists do with these universes. Namely, with the universes, it would seem to suffice for almost all category-theoretic purposes, if a given universe U were merely a model of ZFC, rather than Vκ for an inaccessible cardinal κ. The difference is that U is merely a model of the Power set axiom, rather than actually being closed under the true power sets (and similarly using Replacement in place of regularity). The weakening of AU I have in mind is the axiom that asserts that every set is an element of a transitive model of ZFC. This assertion is strictly weaker in consistency strength thatn even a single inaccessible cardinal. One can get much lower, if one weakens the concept of universe to just a fragment of ZFC. Then one could arrive at a version of AU that was actually provable in ZFC, but which could be used for most all of the applications in cateogory theory to my knowledge. In this sense, ZFC itself is a kind of large cardinal axiom relative to the weaker fragments of ZFC.
I'm inclined to agree that "if you precisely identify the notion we're talking about (like the integers, set theory), then these pathological models don't exist." The problem is that it's not so easy to precisely identify such structures.
The usual approach to precisely identifying a structure is to write down its essential properties, the axioms governing it. To make use of such axioms, we need to derive consequences from them, and here we find ourselves in a dilemma. On the one hand, there is a perfectly clear notion of logical deduction in the context of first-order logic. On the other hand, the L"owenheim-Skolem-Tarski theorem guarantees that, in the context of first-order logic, there will be unintended models of the axioms (as long as the intended model was infinite). So first-order logic does not accomplish what is wanted.
So let's use second- (or higher-)order logic insterad. (Here you can quantify over subsets of the structure, and those quantified variables are assumed to really range over all subsets, not just, say, definable ones.) Now structures like the integers and the reals can be uniquely specified. But there is no complete deductive system for second-order logic. (More precisely, the set of valid second-order sentences is not recursively enumerable.) Furthermore, the intended meaning of second-order quantifiers depends on the general notion of "set," which is one of the concepts that we were hoping to precisely specify.
So the bottom line, in my opinion, is that, while we might want to build mathematics on the basis of unique specifications of the relevant structures, it simply can't be done, at least not if we want the specifications to contain actual information about the structures (as opposed to just saying "I mean the genuine integers, you know") and to be able to deduce the logical consequences of that information.
Best Answer
It might help to understand look at how the fifth postulate is proved independent of Euclid's other axioms: One constructs a model, such as the Poincare disc, where the axioms can be given new interpretations. So the word LINE now means "arc perpindicular to the boundary of the disc", the word CONGRUENT now means "related by a Mobius transformation fixing the boundary of the disc" and so forth. One then checks that, if you take each axiom and replace the capitalized words by the quoted strings, the axiom remains true, except for the fifth postulate, which becomes false.
Similarly, to show that AC is independent of ZF, one works inside ZFC and builds a model where the axioms of ZF, suitably reinterpreted, stay true but where choice is false. The technical tool used to build that model is called forcing. I don't really understand it, but Timothy Chow's introduction is the closest I have come to doing so.