(Thanks for the ping, @Pierre-Yves Gaillard...)
As in Pierre-Yves Gaillard's answer, I think that the usual formalization of "Yoneda's Lemma" does not literally yield the exactness assertion quoted. However, if one tolerates a certain amount of figurative reference, it's not completely ridiculous to refer to using $X\rightarrow Hom(X,-)$, and inferences about the original category from the image, as Yoneda-ish. After all, isn't that why we care about the Yoneda maps?
As Pierre-Yves G notes, and as in @Agusti Roig's elaboration of the key point of the argument, there are further ingredients in play, such as additivity or abelian-ness, which were built into the example scenario my note treated, so I did not have to name them. Thus, the pure Yoneda Lemma is not directly addressing all the issues. However, the idea to "consider" $X\rightarrow Hom(X,-)$ is the watershed, I think. If we take the simplest illustrative situation, abelian groups, so that ("by luck") the Hom(,)'s are in the same category, etc., one can immediately proceed to "do the thing" with no formalism whatsoever. One of the nice didactic points is that we really, really do need the "naturality" to know that the squares commute, to reach the conclusion. This is the simplest illustration I know (apart from the usual algebraic topology examples) of the on-the-street genuine content of "naturality", as opposed to the all-too-common cocktail-party misunderstanding of it as "not making arbitrary choices", etc.
But, yes, then the narrative was not connected to any formalism, either. The features of $X\rightarrow Hom(X,-)$ that made things work were not abstracted or formalized.
I'd plead guilty to "formal incorrectness", but with extenuating circumstances, as follows. Namely, the point of the linked-to note was to illustrate the immediate utility of $X\rightarrow Hom(X,-)$ to very tangible issues, such as showing right-exactness of $-\otimes X$, by seeing that it is a left adjoint, and showing left adjoints (with additivity...) are right exact, for very general reasons. The intended audience had/has little prior acquaintance with category theory or homological algebra, and that note was intended as an advertisement or promotion of such ideas, without formalizing those ideas enough to name them precisely. If I recall correctly, in particular, someone had been wrangling with a direct argument for the right exactness of tensor-product, and had gotten bogged-down in details that shouldn't matter, and I was trying to make the case to them that the causality in the situation could (without paying too heavy a price) be better understood as being about adjoint functors, etc. To make that case, there was absolutely no room for any set-up. At the same time, the name-dropping seemed appropriate on philosophical grounds.
I am in no sense "a category theorist" or "homological algebraist", but do have some appreciation for the widespread usefulness of the ideas, "even" without formalism or abstraction of categorical notions. Indeed, I would claim that the ideas "are there", whether one recognizes/formalizes them or not, and that recognizing them is immediately useful in the most pragmatic sense. Here I am playing against a common belief that one "chooses", or not, to "do" category theory or homological algebra, and that ignoring those ideas is a viable, completely sensible option.
In that context, perhaps over-selling "Yoneda" by throwing in a few further ingredients is an excess that can be forgiven? (Not quite as bad as a "stone soup" scenario.)
I do also admit to an ever-waning interest in formalization, especially definitions. Pithy examples seem to me much better, even without a pre-existing name for the phenomenon illustrated. One "problem" with this is that naturally-occurring illustrative examples often are not "pure", in the sense that more than the single phenomenon is involved. E.g., if I say I'm illustrating Yoneda by the exactness argument at hand, the illustration is not "pure", and there is potential confusion, indeed. On the other hand, if I'm trying to convince someone of the virtues of an idea (the $X\rightarrow Hom(X,-)$), it would be unwise to deliberately ignore the "extra" features they actually cared about.
In summary: indeed, Yoneda does not literally entail that exactness conclusion, I think. Nor need it be invoked in the simple case at hand. Rather some aspects of a special case of Yoneda are being proved directly, along with some additional bits to address the problem at hand.
Edit: in response to @Bruno Stonek's follow-up question, the "naturality" is the obvious, essentially trivial assertion that exactness of the top row entails exactness of the bottom row in the following:
$$
\matrix{
0 & \rightarrow & Hom(LX,A) & \rightarrow & Hom(LX,B) & \rightarrow & Hom(LX,C)
\cr
& & \downarrow & & \downarrow & & \downarrow
\cr
0 & \rightarrow & Hom(X,RA) & \rightarrow & Hom(X,RB) & \rightarrow & Hom(X,RC)
}
$$
where the vertical arrows are isomorphisms, and $L,R$ are left and right adjoint to each other. That is, without naturality (or whatever name one likes), the vertical isomorphisms would not necessarily assure that the squares commute.
Another Edit: in response to Bruno Stonek's further "tangential" question... The "additivity" or some similar qualifier is necessary for "exactness" to make sense at all. In familiar concrete categories where we'd have any impulse to mention it, such as categories of modules over some ring, it's just structure we're used-to. Not every category behaves this way, of course. Sufficient notions to talk about kernels and cokernels can obviously be axiomatized...
Here's one possible answer to this question.
Let's take the viewpoint that functors are representations of categories.
First, why is this sensible?
Well, recall that categories are generalizations of monoids (and consequently groups as well), since a one object category is the same thing as a monoid.
If $M$ is a monoid, then we can define a category, $C$, with one object, $*$, hom set $C(*,*)=M$, and unit and composition given by the unit and multiplication in $M$. Conversely, given a one object category $C$, $C(*,*)$ is a monoid with composition as multiplication, and these constructions are inverse to each other.
From now on, if $M$ is a monoid, or $G$ is a group, I'll write $BM$ or $BG$ for the corresponding one object category.
Now, what about functors? Well, what are functors $[BG,k\newcommand\Vect{\text{-}\mathbf{Vect}}\Vect]$?
Well, we need to pick a vector space $V$ to send $*$ to, and we need to pick a monoid homomorphism $G\to \newcommand\End{\operatorname{End}}\End V$. Since $G$ is a group, this is equivalent to a group homomorphism $G\to \operatorname{GL}(V)$. In other words, functors from $BG$ to $k\Vect$ are exactly the same as linear group representations, and you can check that natural transformations of functors correspond exactly to the $G$-equivariant linear maps.
Similarly, when we replace $k\Vect$ with $\newcommand\Ab{\mathbf{Ab}}\Ab$, or $\newcommand\Set{\mathbf{Set}}\Set$, we get $G$-modules and $G$-sets respectively.
Specifically, these are all left $G$-actions, since a functor $F:BG\to \Set$ must preserve composition, so
$F(gh)=F(g)F(h)$, and we define $g\cdot x$ by $F(g)(x)$. Thus $(gh)\cdot x = g\cdot (h\cdot x))$.
A contravariant functor $\newcommand\op{\text{op}}BG^\op\to \Set$ gives a right $G$-action, since now
$F(gh)=F(h)F(g)$, so if we define $x\cdot g = F(g)(x)$, then we have
$$x\cdot (gh) =F(gh)(x) = F(h)F(g)x = F(h)(x\cdot g) = (x\cdot g)\cdot h.$$
Thus we should think of covariant functors $[C,\Set]$ as left $C$-actions in $\Set$,
and we should think of contravariant functors $[C^\op,\Set]$ as right $C$-actions in $\Set$.
Yoneda Lemma in Context
Representable presheaves now correspond to free objects in a single variable in the following sense.
The Yoneda lemma is that we have a natural isomorphism
$$
[C^\op,\Set](C(-,A),F)\simeq F(A)\simeq \Set(*,F(A)).
$$
In other words, $C(-,A)$ looks a lot like the left adjoint to the "forgetful" functor that sends a presheaf $F$ to its evaluation at $A$, $F(A)$, but evaluated on the singleton set $*$.
In fact, we can turn $C(-,A)$ into a full left adjoint by noting that
$$\Set(S,F(A)) \simeq \prod_{s\in S} F(A) \simeq \prod_{s\in S}[C^\op,\Set](C(-,A),F)
\simeq [C^\op,\Set](\coprod_{s\in S} C(-,A), F),$$
and $\coprod_{s\in S} C(-,A)\simeq S\times C(-,A)$.
Thus one way of stating the Yoneda lemma is that $S\mapsto S\times C(-,A)$ is left adjoint to the evaluation at $A$ functor (in the sense that the two statements are equivalent via a short proof). Incidentally, there is also a right adjoint to the evaluation at $A$ functor, see here for the argument.
Relating this back to more familiar notions
First thing to notice in this viewpoint is that we now have notions of "free on an object" rather than just "free." I.e., I tend to think of $C(-,A)$ as being the free presheaf in one variable on $A$ (this is not standard terminology, just how I think of it).
Now we should be careful, a free object isn't just an object, it's an object and a basis. In this case, our basis (element that freely generates the presheaf) is the identity element $1_A$.
Thinking about it this way, the proof of the Yoneda lemma should hopefully be more intuitive. After all, the proof of the Yoneda lemma is the following:
$C(-,A)$ is generated by $1_A$, since $f^*1_A=f$, for any $f\in C(B,A)$, so natural transformations $C(-,A)$ to $F$ are uniquely determined by where they send $1_A$. (Analogous to saying $1_A$ spans $C(-,A)$). Moreover, any choice $\alpha\in F(A)$ of where to send $1_A$ is valid, since we can define a natural transformation by "extending linearly" $f=f^*1_A \mapsto f^*\alpha$ (this is analogous to saying $1_A$ is linearly independent, or forms a basis).
The covariant version of the Yoneda lemma is the exact same idea, except that we are now working with left representations of our category.
Examples of the Yoneda lemma in more familiar contexts
Consider the one object category $BG$, then the Yoneda lemma says that
the right regular representation of $G$ is the free right $G$-set in one variable (with the basis element being the identity, $1_G$).
(The free one in $n$-variables is the disjoint union of $n$ copies of the right regular representation.)
The embedding statement is now that $G$ can be embedded into $\operatorname{Sym}(G)$ via $g\mapsto -\cdot g$.
This also works in enriched contexts. A ring is precisely a one object category enriched in abelian groups, and the Yoneda lemma in this context says that the right action of $R$ on itself (often denoted $R_R$) is the free right $R$-module in one variable, with the basis being the unit element $1_R$.
(The free one in $n$-variables is now the direct sum of $n$ copies of $R_R$)
The embedding statement here is that $R$ can be embedded into the endomorphism ring of its underlying abelian group via $r\mapsto (-\cdot r)$.
Best Answer
Some elaboration on Dylan Moreland's comment is in order. Consider the gadget $\text{GL}_n(-)$. What sort of gadget is this, exactly? To every commutative ring $R$, it assigns a group $\text{GL}_n(R)$ of $n \times n$ invertible matrices over $R$. But there's more: to every morphism $R \to S$ of commutative rings, it assigns a morphism $\text{GL}_n(R) \to \text{GL}_n(S)$ in the obvious way, and this assignment satisfies the obvious compatibility conditions. That is, $\text{GL}_n(-)$ defines a functor $$\text{GL}_n(-) : \text{CRing} \to \text{Grp}.$$
Composing this functor with the forgetful functor $\text{Grp} \to \text{Set}$ gives a functor which turns out to be representable by the ring $$\mathbb{Z}[x_{ij} : 1 \le i, j \le n, y]/(y \det_{1 \le i, j \le n} x_{ij} - 1).$$
Now, this ring itself only defines a functor $\text{CRing} \to \text{Set}$. What extra structure do we need to recover the fact that we actually have a functor into $\text{Grp}$? Well, for every ring $R$ we have maps $$e : 1 \to \text{GL}_n(R)$$ $$m : \text{GL}_n(R) \times \text{GL}_n(R) \to \text{GL}_n(R)$$ $$i : \text{GL}_n(R) \to \text{GL}_n(R)$$
satisfying various axioms coming from the ordinary group operations on $\text{GL}_n(R)$. These maps are all natural transformations of the corresponding functors, all of which are representable, so by the Yoneda lemma they come from morphisms in $\text{CRing}$ itself. These morphisms endow the ring above with the extra structure of a commutative Hopf algebra, which is equivalent to endowing its spectrum with the extra structure of a group object in the category of schemes, or an affine group scheme.
In other words, in a category with finite products, saying that an object $G$ has the property that $\text{Hom}(-, G)$ is endowed with a natural group structure in the ordinary set-theoretic sense is equivalent to saying that $G$ itself is endowed with a group structure in a category-theoretic sense. I discuss these ideas in some more detail, using a simpler group scheme, in this blog post.