Here is the construction in the case of limits.
A diagram $(X, a) : \mathsf{J} \to \int F$ consists of a diagram $X = \Pi \cdot (X, a) : \mathsf{J} \to \mathsf{C}$ along with a cone $a : * \to F\cdot X$ in $\mathsf{Set}$. Suppose that $X : \mathsf{J} \to \int F \to \mathsf{C}$ has a limit cone $\eta : \lim X \to X$ and that $F(\eta) : F(\lim X) \to F \cdot X$ is also a limit cone.
We'll show that if $\eta$ has a lift then the lift is unique. Suppose that $\eta$ lifts to a cone $\tilde{\eta} : z \to (X, a)$. Since $\Pi$ acts as the identity on arrows we must have $\tilde{\eta}_{j} = \eta_{j}$ for all $j$ in $\mathsf{J}$. So we must have $\Pi(z) = \lim X$, meaning that $z$ is of the form $(\lim X, x)$, where $x$ is an element of $F(\lim X)$. Since $\tilde{\eta} : (\lim X, x) \to (X, a)$ is a cone in $\int F$ we must have $F(\eta_{j})(x) = a_{j}$ for all $j \in \mathsf{J}$. And since $F(\eta)$ is a limit cone there is a unique element $x$ of $F(\lim X)$ satisfying these equations, so there is at most one lift of $\eta : \lim X \to X$ to a cone in $\int F$.
So now to show that $\Pi$ strictly creates limits we just have to check that this $\tilde{\eta} : (\lim X, x) \to (X, a)$ is a cone in $\int F$ and that it is limiting. I'll leave it to you to do the checking.
When $(G,F)$ is monadic, then $G\circ F$ preserves the same colimits as $G$ : in one direction it's easy to see, and in the other it's just checking that for any monad $T$, the forgetful functor from $T$-algebras preserves these colimits (as it is then "the same" as $G$), and that's done by building the colimits by hand.
For nonmonadic adjunctions, the question is not so easy; perhaps there is an answer (I'd like to see it) but I don't know it. If you look at e.g. the adjunction between the category of free abelian groups and the category of sets (the restriction of the usual one between abelian groups and sets) and the colimit of the directed diagram $\mathbb{Z\overset{2}\to Z \overset{3}\to Z \overset{5}\to \dots}$, its colimit in free abelian groups is the free group on no generators (this is the case because if we have a cocone to a free abelian group $F(X)$, then the image of the first $\mathbb Z$ is divisible by all integers, but $F(X)$ is residually finite so that's not possible unless that map is $0$, the same for the rest), whereas its colimit in abelian groups (and hence in sets) is $\mathbb Q$, so it's not preserved under the forgetful functor.
On the other hand, differential graded Lie or commutative algebras are (I think) monadic over differential graded modules, so it's equivalent to check that the forgetful functor preserves $\lambda$-filtered colimits; and this should be easier.
ADDED : Note first that the direction we're interested in here is the easy one : of $G$ preserves $I$-shaped colimits, then so does $G\circ F$: this simply follows as $F$ preserves all colimits. The point about monadic adjunctions was simply to say that it was in fact equivalent in our situation.
Here's a proof of that claim. It is clear that it suffices to show that for a monad $T$ on a category $C$, the forgetful functor $U: C^T \to C$ preserves all colimits that $T$ does (where $C^T$ is the category of algebras of $T$). For that we explicitly build colimits in $C^T$ : let $I$ be a category such that $C$ has $I$-shaped colimits and $T$ preserves them, and let $D: I\to C^T$ be a diagram.
Let $L = \mathrm{colim}_I U\circ D$ (which exists by assumption) and $\iota_i D(i)\to L$ the canonical maps (I'm abusing notation and writing $D(i)$ for the object of $C$ as well as for the $T$-algebra). The point is to try and construct a $T$-algebra structure on $L$ such that the $\iota_i$ are $T$-morphisms.
But now we also have a natural isomorphism $\mathrm{colim}_I TD(i) \to TL$ : the natural map is always defined, and it is an isomorphism because by assumption $T$ preserves colimits. So to define a map $TL\to L$ it suffices to define compatible maps $TD(i) \to L$. But that's easy : just composte $TD(i)\to D(i)\to L$ where the first map is the structure map for $D(i)$. These maps are compatible precisely because $D$ is a functor into $T$ -algebras, so the squares $$\require{AMScd}\begin{CD}TD(i) @>>> D(i) \\
@V{T(f)}VV @V{f}VV \\
TD(j) @>>> D(j)\end{CD}$$
commute for all $f:i\to j$ in $I$. Therefore this system of maps allows one to define $\mathrm{colim}_I TD(i)\to L$ so $TL\to L$.
Now the annoying bookkeeping comes : we have to check that this does define a $T$-algebra structure; and that this structure is in fact a colimit of $D$ in $C^T$. This is simply using uniqueness in universal properties, naturality of certain morphisms and preservation of $I$-colimits by $T$ a bunch of times; if you're not convinced you should do it as an exercise.
A full proof (no details skipped) should be in Borceux's Handbook of Categorical algebra (volume 1 or 2)
Best Answer
Yes $U:\mathrm{Ring}\to\mathrm{Set}$ preserves directed colimits. The directed colimit of the underlying set of a directed family $(R_i)$ of rings is endowed with a ring structure in which the ring operations on a sequence $(r_1,...,r_k)$ of elements are defined by mapping all the $r_i$ into some ring $R_k$ in which they are all represented, then applying the operations of $R_k$. It is straightforward to verify that this ring satisfies the universal property of the direct limit, and that the same argument applies to any category of algebras and finitary operations, and to filtered colimits as well as directed colimits.
However, the same does not hold true for arbitrary monads. For instance, the monad $\beta$ which sends a set $A$ to the set of ultrafilters on $A$ has as category of algebras the category of compact Hausdorff spaces, and the forgetful functor from $\mathrm{CompHaus}$ to $\mathrm{Set}$ does not preserve directed colimits. For instance, the colimit of the sequence of discrete spaces $\{1,...,n\}$ in $\mathrm{CompHaus}$ is the Stone-Cech compactification of $\mathbb N$, a rare natural example of a space with cardinality greater than that of the continuum, while the colimit of the underlying sets is good old $\mathbb{N}$.
Any monad for algebras with infinitary operations, such as the monad for lattices equipped with countable suprema, will similarly fail to preserve directed colimits.