From Vakil's notes:
The notion of colimit is defined for diagrams $D:I\to\textbf{Set}$. The colimit of $D:I\to\textbf{Set}$ is the limit of $D^{op}:I^{op}\to \mathbf{Set}^{op}$. What exactly is the diagram $D:I\to\textbf{Set}$ in 2.2.5? Well, I guess $I$ is given (it's the category whose objects are all open sets containing $p$), and I thought $D$ acts on objects by sending $U$ to $\mathscr F(U)$. But $\mathscr F$ is a contravariant functor, so the $D$ that I defined should be a functor with domain $I^{op}$. How to sort out this mess?
Now, to see that this definition of stalk is equivalent to the non-categorical one, I guess one needs to explicitly describe the colimit. The (vertex of the) colimit (cocone) in this case is $\sum_I \mathscr F (U)$ modulo the equivalence relation generated by something like $\{(f,res_{V,U}(f)): f\in \mathscr F (V), U\subset V\text{ in } I \}$. How is this the same as the set of equivalence classes of pairs of the form $(g,U)$? There is a "type" mismatch.
Best Answer
Consider the partial order $\mathcal{O}_X^{op}$, expressed as a category.
That is, $U \to V$ is the set $\{0 \mid V \subseteq U\}$. There is exactly 1 arrow $U \to V$ if and only if $V \subseteq U$, and there is no arrow $U \to V$ otherwise.
Now $\mathscr{F}$ is a functor $\mathcal{O}_X^{op} \to Sets$.
Consider the full subcategory $Neigh_p$ of $\mathcal{O}_X^{op}$, which contains only those open sets that are neighbourhoods of the point $p \in X$. Let $i : Neigh_p \to \mathcal{O}_X^{op}$ be the fully faithful inclusion functor.
Then the colimit is taken over the diagram $\mathscr{F} \circ i : Neigh_p \to Sets$.
To show that this definition is equivalent to the non-categorical one, you need to prove the universal property of the colimit for the non-categorical definition.