Let $I$ be a set and $\mathcal{U}$ an ultrafilter on $I$. Let $(X_i)_{i \in I}$ be an $I$-indexed family of sets. The ultraproduct of the family $(X_i)$ with respect to $\mathcal{U}$ is, everyone agrees, another set. But which set is it? There are two different definitions, and they sometimes give different results.
For the sake of discussion, I'll call them "Type 1" and "Type 2" ultraproducts.
Type 1 $\ $ The type 1 ultraproduct of $(X_i)_{i \in I}$ with respect to $\mathcal{U}$ is
$$
\Bigl( \prod_{i \in I} X_i \Bigr) \Bigl/ \sim
$$
where
$$
(x_i)_{i \in I} \sim (x'_i)_{i \in I} \iff \{ i \in I: x_i = x'_i \} \in \mathcal{U}.
$$
Type 2 $\ $ View the poset $(\mathcal{U}, \subseteq)$ as a category. The type 2 ultraproduct of $(X_i)_{i \in I}$ with respect to $\mathcal{U}$ is the colimit of the functor $(\mathcal{U}, \subseteq)^{\text{op}} \to \mathbf{Set}$ defined on objects by
$$
J \mapsto \prod_{j \in J} X_j
$$
and on maps by projection. Explicitly, then, the Type 2 ultraproduct is
$$
\Bigl( \coprod_{J \in \mathcal{U}} \prod_{j \in J} X_j \Bigr) \Bigl/ \approx
$$
where
$$
(x_j)_{j \in J} \approx (x'_k)_{k \in K} \iff \{ i \in J \cap K: x_i = x'_i \} \in \mathcal{U}.
$$
The difference $\ $ The two types of ultraproduct are the same if either none of the sets $X_i$ are empty or almost all of them are empty. But in the remaining case, where at least one $X_i$ is empty but the set of such $i$ is not large enough to belong to $\mathcal{U}$, they're different: the Type 1 ultraproduct is empty but the Type 2 ultraproduct is not.
The question $\ $ I've read in a couple of texts (both coming from the point of view of categorical logic) that the Type 2 ultraproduct is really the right one. But why? On what criteria is Type 2 judged to be better than Type 1?
A vague guess at an answer $\ $ I think I can guess very roughly what's going on. There's been a tradition in logic — maybe dying out now? — of taking all structures to be nonempty by definition. But when you move to the more general setting of categorical logic, that's no longer a satisfactory approach. Although in the category of sets, there's just a single object with no elements, in many other categories, there are lots of interesting objects with no (global) elements: e.g. there are lots of interesting sheaves with no global sections.
So categorical logic sometimes involves a recasting of classical, set-based logic, in order to handle empty sets/types satisfactorily. I imagine that something of the sort is going on here. (I only defined ultraproducts of sets, but you could of course define ultraproducts of objects of any other sufficiently complete category.) But still, I don't see clearly why Type 2 is the right choice.
See also This question of Joel David Hamkins, and its responses.
Best Answer
Since the question mentions texts with a categorical-logic point of view, which tends to connect with constructive logic, and since Francois's answer (with which I completely agree) adopts the viewpoint of classical logic, it seems worthwhile to add the following. As Francois said, empty domains are not very interesting, so it's natural to ignore them. But in constructive logic, it may not be decidable whether a domain is empty. That is, given a set $D$, one cannot generally assert that either $D$ is empty (and therefore safe to ignore) or $D$ has a member. To prove that the two definitions of ultraproduct give the same (i.e., canonically isomorphic) results, one needs that the domains of the structures have members (and in fact, one needs a choice function for these domains). If one is working in constructive model theory (but nevertheless managed somehow to get hold of an ultrafilter), then it's a bad idea to try to ignore empty structures, and so one must pay attention to the difference between the two definitions of ultraproduct.