Some very involved necessary and sufficient conditions are found in a paper of Foltz (in French). Some observations on his paper:
An elementary observation (Proposition 3, section 1, p. F 12): $I$-colimits commute in $\mathrm{Set}$ with $P$-limits iff $I$-limits commute with discrete $\pi_0(P)$-colimits and also with $P'$-colimits for each connected component $P'$ of $P$. Foltz then analyzes separately the cases of $P$ discrete and of $P$ connected.
He separately analyzes the conditions that the canonical comparison map be always injective and that it be always surjective.
He treats some examples of interest at the end, including the colimits that commute in $\mathrm{Set}$ with pullbacks and those that commute in $\mathrm{Set}$ with equalizers. But it doesn't appear that he discusses how to recover characterizations of filtered or sifted limits.
Foltz's criteria are expressed in terms of certain certain subdivision categories, and a lot of zig-zags. Unfortunately, he doesn't discuss how to relate his criteria to other more familiar ones, such as the finality of certain diagonal functors. But it might be possible to convert his criteria into such forms.
Some things are known about the general phenomenon of limits commuting with colimits:
- Albert and Kelly's "The Closure of a Class of Colimits" discusses which limit-weights commute in $\mathrm{Set}$ with all the colimit-weights that a given class commutes with -- which is sort of the "square" of the commutation relation you're interested in. This is what Albert and Kelly call the "closure" of a class of colimits, and nowadays is typically referred to as the saturation.
- There are also some good notes by Kelly and Schmitt which discuss the formal aspects of the situation, which is enough to gain some meaningful insight into the important case of absolute colimits -- those which commute with every limit.
Both of these papers are written in the context of enriched categories, which means they don't provide terribly specific information about the case of $\mathrm{Set}$-enrichment, but at least clarify the formal situation.
More specifically, as Mike Shulman notes, you might want to take a look at the
ABLR paper, available from Steve Lack's website. They use a condition on a class of limit weights $\mathbb{D}$ that they call "soundness." In fact, soundness is explicitly a simplifying assumption about which colimits commute with $\mathbb{D}$-limits in $\mathrm{Set}$. All the examples which are well-known (like finite/filtered and finite-discrete/sifted) satisfy soundness; it seems to account for why they're so nice to work with.
Some further work has been done on developing the theory of these "sound doctrines", especially by Claudia Centazzo; Lack and Rosicky's "On the notion of Lawvere Theory" also starts to consider what the enriched case might look like.
But very little seems to be known about which "doctrines" (classes of limit-weights) are sound in general. In fact, the only examples given by ABLR of non-sound doctrines are the doctrine of pullbacks, and the doctrine of pullbacks + terminal objects -- neither of which is saturated! The saturation of the latter is, of course, all finite limits, which is sound. The conical saturation of pullbacks is the class of simply-connected and finitely-presentable categories, as discovered by Paré, which is not sound -- this can be seen by adapting ABLR's argument concerning pullbacks (Example 2.3.vii).
Linked references:
- François Foltz, Sur la commutation des limites, Diagrammes 1981
- Kelly and Schmitt, Notes on enriched categories with colimits of some class, Theory and Applications of Categories 2005
- Adámek, Borceux, Lack, Rosický, A classification of accessible categories, Journal of Pure and Applied Algebra, 2002.
- Paré, Simply connected limits, Canadian Journal of Mathematics, 1990
It should be noted that already in $\mathbf{Set}$, the free functor $\mathbf Z^{(-)} \colon \mathbf{Set} \to \mathbf{Ab}$ does not preserve cofiltered limits. For a cofiltered diagram $D \colon \mathcal I \to \mathbf{Set}$, write $S_i$ for its value at $i \in \mathcal I$, write $S$ for its limit, and write $\pi_i \colon S \to S_i$ for the canonical projection.
There is always a map
\begin{align*}
\phi \colon \mathbf Z^{(S)} &\to \lim_\leftarrow \mathbf Z^{(S_i)}\\
\sum_{k=1}^n n_k s_k &\mapsto \left( \sum_{k=1}^n n_k\pi_i(s_k) \right)_i.
\end{align*}
Denote its $i^{\operatorname{th}}$ component by $\phi_i$. For a set $X$ and an element $z=\sum_{k=1}^n n_k x_k \in \mathbf Z^{(X)}$ with $x_k \neq x_{k'}$ for $k \neq k'$, write $\operatorname{Supp}(z)$ for $\{x_1,\ldots,x_n\}$, and denote by $|z|$ its cardinality. If $f \colon X \to Y$ is a map, we denote the induced map $\mathbf Z^{(X)} \to \mathbf Z^{(Y)}$ by $f$ as well (by abuse of notation). For $z \in \mathbf Z^{(X)}$, we have $\operatorname{Supp}(f(z)) \subseteq f(\operatorname{Supp}(z))$, so $|f(z)| \leq |z|$ with equality if and only if $f_* \colon \operatorname{Supp}(z) \to \operatorname{Supp}(f(z))$ is a bijection.
Lemma. The map $\phi$ is injective, and its image consists of those $(x_i)_{i \in \mathcal I}$ for which there exists $n \in \mathbf Z$ with $|x_i| \leq n$ for all $i \in \mathcal I$.
In other words, the image consists of the sequences $(x_i)_i$ of bounded support.
Proof. For injectivity, if $x = \sum_{k=1}^n n_ks_k \in \ker(\phi)$ is such that $s_k \neq s_{k'}$ for $k \neq k'$, then there exists $i \in \mathcal I$ such that $\pi_i(s_k) \neq \pi_i(s_{k'})$ for $k \neq k'$ (here we use that the sum is finite and that $\mathcal I$ is cofiltered). Then $\phi_i(x) = \sum_{k=1}^n n_k\pi_i(s_k)$ is zero by assumption, so all $n_k$ are zero.
For the image, it is clear that $|\phi_i(x)| \leq n$ for all $i \in \mathcal I$ if $x \in \mathbf Z^{(S)}$ has $|x| = n$. Conversely, if $|x_i| \leq n$ for all $i \in \mathcal I$, then decreasing $n$ if necessary, we may assume $|x_{i_0}| = n$ for some $i_0 \in \mathcal I$. Then $|x_i| = n$ for all $f \colon i \to i_0$ since $n = |x_{i_0}| = |D(f)(x_i)| \leq |x_i| \leq n$. Replacing $\mathcal I$ by the coinitial segment $\mathcal I/i_0$ we may therefore assume $|x_i| = n$ for all $i \in \mathcal I$.
Constancy of $|x_i|$ means that for every morphism $f \colon i \to j$ in $\mathcal I$, the map $f_* \colon \operatorname{Supp}(x_i) \to \operatorname{Supp}(x_j)$ is a bijection. Setting $T = \lim\limits_\leftarrow \operatorname{Supp}(x_i)$, we see that each projection $\pi_i \colon T \to \operatorname{Supp}(x_i)$ is a bijection. Functoriality of the limit gives an injection $T \hookrightarrow S$, giving elements $s_1,\ldots,s_n \in S$ such that $\pi_i(\{s_1,\ldots,s_n\}) = \operatorname{Supp}(x_i)$ for all $i \in \mathcal I$. The coefficients must also be constant under the bijections $\operatorname{Supp}(x_i) \to \operatorname{Supp}(x_j)$, so we get an element $x = \sum_{k=1}^n n_ks_k \in \mathbf Z^{(S)}$ with $\phi(x) = (x_i)_i$. $\square$
Example. An example where $\phi$ is not surjective: let $S = \mathbf Z_3$ with $S_i = \mathbf Z/3^i$. Define the element $(x_i)_i \in \prod_i \mathbf Z^{(S_i)}$ where $x_i \in \mathbf Z^{(S_i)} = \mathbf Z^{S_i}$ has coordinates (for $k \in \{0,\ldots,3^i-1\}$) given by
$$x_{i,k} = \begin{cases} 1, & \text{the first $3$-adic digit of } k \text{ is } 1, \\ -1, & \text{the first $3$-adic digit of } k \text{ is } 2, \\ 0, & k=0.\end{cases}$$
These form an element of $\lim\limits_\leftarrow \mathbf Z^{(S_i)}$: any fibre of $\mathbf Z/3^{i+1} \to \mathbf Z/3^i$ above $k \in \{0,\ldots,3^i-1\}$ consists of $\{k,3^i+k,2 \cdot 3^i+k\}$, of which $x_{i+1,k} = x_{i,k}$ and the others are $1$ and $-1$ since $3^i+k$ starts on $1$ and $2 \cdot 3^i+k$ starts on $2$.
Since $\operatorname{Supp}(x_i) = S_i \setminus \{0\}$, we see that $(x_i)_i$ does not have bounded support, hence is not in the image of $\phi$.
Corollary. For any presheaf topos $\mathbf T = \mathbf{PSh}(\mathscr C) = [\mathscr C^{\operatorname{op}},\mathbf{Set}]$ on a small nonempty category $\mathscr C$, the free functor $\mathbf Z^{(-)} \colon \mathbf T \to \mathbf{Ab}(\mathbf T)$ does not preserve cofiltered limits.
Proof. Let $f \colon \mathscr C \to *$ be the map to the terminal category $*$, on which $\mathbf{PSh}(*) = \mathbf{Set}$. Note that $f$ has a section $g$ since $\mathscr C$ is nonempty. We saw above that in $\mathbf{Set}$, the free functor $\mathbf Z^{(-)}$ does not preserve cofiltered limits. The pullback $f^* \colon \mathbf{Set} \to \mathbf{PSh}(\mathscr C)$ takes a set $S$ to the constant sheaf $\underline{S}$ on $\mathscr C$. Since limits, colimits, and free abelian group objects in presheaf categories are pointwise, $f^*$ commutes with formation of limits, colimits, and free abelian group objects. Thus pulling back everything along $f^*$ gives the natural map $\underline{\mathbf Z}^{(\underline S)} \to \lim\limits_\leftarrow \underline{\mathbf Z}^{(\underline S_i)}$, which is not an isomorphism since it isn't after applying the section $g^*$ (this is just evaluation at an object $g(*) \in \mathscr C$). $\square$
The class of topoi where the free functor does not commute with cofiltered limits is probably much larger still. For instance, my example does not include condensed sets, which is close to a presheaf topos but not quite. I'm not even sure what a topos would look like where these do commute!
Best Answer
The objects we are taking the limit over in the left side are $(i_j \in I_j)_j$, i.e. tuples of, for each $j\in J$, an element $i_j$ of $I_j$. These are the same as elements of $\prod_{j\in J}$.
Thus, the left side is a limit over $\prod_{j\in J} I_j$.
On the other hand, you have defined $I_j = I$ for all $j$ and then taken the limit over $I$. Of course in this case $I$ embeds into $\prod_{j\in J} I$ as a "diagonal" subset, but not every element is bounded by an element on the diagonal, unless $I$ is finite.