This is a crosspost of this MSE question.
Every so often, when reading notes online or skimming through books, the category of elements and the Grothendieck construction pop up. I don't know anything about the Grothendieck construction, and I don't understand the significance of its special case – the category of elements. This is probably going to be the first in a series of questions about the Grothendieck construction (I'll interlink with later ones if/when I ask them).
My main motivation for asking this question is the excerpt below from Borceux vol II, sec 6.6. I have tried to go into detail about how I don't understand the significance of the category of elements in this context.
The first observation for enriching the notion of limit is the fact in many fundamental results of the previous chapters, we had to deal with a situation like in diagram 6.3.6, where $F,G$ are functors and $\mathsf{Elts}(G)$ is the category of elements of $G$; the limit considered was that of $F\circ \phi _G$ […]. Restricting one's attention to limits of the form $\varprojlim (F\circ \phi _G)$ is not at all a restriction since, choosing for $G$ the constant functor on the singleton, the category of elements of $G$ is just $\mathscr A$ itself and $\phi _G =1_\mathscr{A}$, so that we recapture the limit of $F$.
But the key observation for enriching the notion of limit is the following fact
Lemma 6.6.1 In the situation which has just been described, there exist bijections natural in $B\in \mathscr B$,
$$\mathsf{Nat}(\varDelta_B,F\circ \phi _G)\cong \mathsf{Nat}(G,\mathscr B(B,F-))$$ where $\varDelta _B:\mathsf{Elts}(G)\longrightarrow \mathscr B$ is the constant functor on the object $B\in \mathscr B$.
From this lemma Borceux concludes that $\varprojlim (F\circ \phi_G)$ exists iff $\exists L\in \mathscr B$ with bijections natural in $B$:
$$\mathsf{Nat}(G,\mathscr B(B,F-))\cong \mathscr B(B,L)$$
And from this, he almost has to write the definition of a weighted limit.
Alternatively, here's an approach I find somewhat more intuitive, but one that does not explicitly mention the category of elements at all.
In classical CT, we define limits using $\varDelta$, which is not available in a general enriched category $\mathsf C$ since $\mathsf C(A,B)$ has no elements to work with. In the unenriched case, one could also try to define $\varDelta$ via the cartesian closed structure of $\mathsf{Cat}$ as a right adjoint of the projection $\pi:\mathsf C\times \mathsf J\longrightarrow \mathsf C$, but this doesn't generalize either since most enriched categories are not cartesian closed. So we're left with trying the representability approach – a limit of $F$ is just a representation of $\mathsf{Nat}(\varDelta-,F)$. Here we can escape the "problems of enrichment" because I think the following holds:
Proposition There's a bijection natural in $X$: $\mathsf{Nat}(\varDelta \ast,\mathsf{Hom}(X,F-))\cong \mathsf{Nat}(\varDelta X,F)$, where $\ast$ is the singleton.
The point is to now only mention the functor $\varDelta \ast$ instead of the general $\varDelta$.
Digression – is the above bijection also natural in $F$? (I think it is.)
Now since $\varprojlim F$ is a representation of $\mathsf{Nat}(\varDelta -,F)$, we can define it as an object with bijections natural in $X$$$\mathsf{Hom}(X,\varprojlim F) \cong \mathsf{Nat}(\varDelta \ast,\mathsf{Hom}(X,F-))$$
Now we have reduced the usage of $\varDelta$ to the functor $\varDelta \ast$ alone. The role of $\varDelta \ast$ as a weight is also easily seen, as each component of natural transformation $\eta _A :\varDelta \ast \Rightarrow \mathsf{Hom}(X,F-)$ picks out an arrow $X\rightarrow FA$, and so replacing
$\varDelta \ast$ by say, $\varDelta (\ast\coprod \ast)$ would give a "double cone". Replacing $\varDelta \ast$ by more general functors $G$ would produce more intricate cones.
This alternative approach is (I think) completely ignorant of the category of elements, and yet manages to give a nice motivation for the definition of a weighted limit. On the other hand, Borceux's approach just poops it out by category-of-elements-magic.
What exactly is happening here? What is the role of the category of elements in this story and why does it effortlessly yield the definition of a weighted limit?
Best Answer
My personal opinion is that the explanations that involve categories of elements (and generally, comma categories) are usually the most natural and intuitive explanations --- one just have to get used to comma categories :-)
In the classical set theory (assuming the axiom of choice) there are two ways to describe indexed families of sets. If we denote by $A$ the set of indices, then an $A$-indexed family may be equivalently described as:
(pointwise description) a function $G \colon A \rightarrow \mathcal{P}(S)$, where $S$ is some set thought of as a universe and $G(a)$ is the collection over $a \in A$; under sufficiently strong foundations one may replace sets $\mathcal{P}(S)$ by a single global universe $\mathcal{U}$ of all (small) sets,
(flat description) a function $g \colon X \rightarrow A$, where for every $a \in A$ the collection over $a$ described by $g$ is $\{x \in X \colon g(x) = a\}$
In categorical terms, these two descriptions may be equally presented as:
(pointwise description) a functor $G \colon A \rightarrow \mathbf{Set}$, where $A = A^{op}$ is a set thought of as a discrete category.
(flat description) a functor $g \colon X \rightarrow A$, where $X$ and $A$ are sets thought as of discrete categories.
A direct categorification of the above two descriptions gives functors $G \colon A \rightarrow \mathbf{Set}$ (or $G \colon A^{op} \rightarrow \mathbf{Set}$ depending on the perspective), and discrete opfibrations $g \colon X \rightarrow A$. A direct categorification of the correspondence between flat and pointwise descriptions is the correspondence established by the Grothendieck construction and its inverse.
Now, in the definition of a $G$-weighted limit the functor $G \colon \mathbb{A} \rightarrow \mathbf{Set}$ says how much every object $A \in \mathbb{A}$ weights in the limit of $F$ --- it weights exactly $G(A)$, it has to have "$G(A)$ uniform (modulo morphisms) copies of itself". If you replace the pointwise indexing $G \colon \mathbb{A} \rightarrow \mathbf{Set}$ by the corresponding flat indexing $\phi_G \colon \mathit{Elts}(G) \rightarrow \mathbb{A}$, then you will explicitly encode in $\mathit{Elts}(G)$ copies of each $A \in \mathbb{A}$. Therefore, one should expect that the limit of $F \circ \phi_G$ is exactly the limit of $F$ weighted by $G$.
The above, of course, explains why in the classical setting it is sufficient to consider classical (conical) limits only. It does not explain, however, why we should define weighted limits in such a way.
Let me offer a more general (and, I think, much simpler) explanation than the usual, which is given in standard textbooks.
We should recall that there is another generalisation of the concept of conical limit --- i.e. Kan extension:
https://en.wikipedia.org/wiki/Kan_extension
The concept of a Kan extension may be defined internally to any 2-category in exactly the same way as in the above link.
The point is that in the 2-category of categories $\mathbf{Cat}$, the limit of a functor $F \colon \mathbb{A} \rightarrow \mathbb{B}$ can be equally defined as the right Kan extension of $F$ along the unique functor to the terminal category $\mathbb{A} \rightarrow 1$. The problem with this definition for general $\mathbb{V}$-enriched categories is the same as has been risen in your question --- the unit category $I$ in $\mathbf{Cat}(\mathbb{V})$ generally may not be terminal, and there are no reasons to believe in the existence of canonical functors $\mathbb{A} \rightarrow I$ from arbitrary $\mathbb{A}$ (another problem is that a general 2-cateogry may not be 2-well pointed, so we miss many limits).
Anyway, one's first attempt could be to define limits weighted by $G \colon \mathbb{A} \rightarrow I$, as right Kan extensions along $G$. However, the problem with this attempt is that weights of the form $G \colon \mathbb{A} \rightarrow I$ are too restrictive for enriched categories. Nonetheless, there is an obvious solution.
The crucial observation is that category $\mathbf{Cat}(\mathbb{V})$ of $\mathbb{V}$-enriched categories embeds (by a 2-functor, let us call it $J$) into a bigger category $\mathbf{Dist}(\mathbb{V})$ of $\mathbb{V}$-enriched categories and distributors. Carrying over one's first attempt from $\mathbf{Cat}(\mathbb{V})$ to $\mathbf{Dist}(\mathbb{V})$, one can define weights as distributors $G \colon \mathbb{A} \nrightarrow I$ (which, recall, are just functors $G \colon \mathbb{A} \rightarrow \mathbb{V}$), and $G$-weighted limits $\mathit{lim}_G F$ of $F$ as right Kan extensions of $J(F) = \hom(-, F(=))$ along $G$ in category $\mathbf{Dist}(\mathbb{V})$. Of course, such right Kan extensions in the category of distributors generally will be distributors of the form $I \nrightarrow \mathbb{B}$. Therefore, we say that limit $\mathit{lim}_G F$ exists if it is represented by an actual functor $I \rightarrow \mathbb{B}$ (which is, of course, tantamount to an object in $\mathbb{B}$) under embedding $J$.
You may easily verify that the above gives the usual definition of weighted limits in the sense of enriched categories.