Let $C$ be a category and $F$ a presheaf on $C$. The Yoneda lemma states that the natural transformations $C(-, A)\Rightarrow F$ are in one-to-one correspondence with the elements of $F(A)$.
To me, this statement feels unmotivated. I know that if we consider the special case $F=C(-, B)$, then (by the Yoneda lemma) $C(-, -)$ yields an embedding (that is, a full and faithful functor) of $C$ into the category of presheaves on $C$. This feels quite motivated, since embedding $C$ into the category of presheaves on $C$ feels a bit like embedding a field $k$ into its algebraic closure $\bar{k}$. Furthermore, the category of presheaves also feels more concrete than the abstract category $C$.
But why is one interested in the general statement where $F$ can be any presheaf whatsoever. Why is this a natural statement? How does one get the idea to consider the Yoneda lemma as stated above?
I have yet another question about the Yoneda lemma. As I said, one has an embedding of $C$ into $[C^\text{op}, \mathbf{Sets}]$ (the category of presheaves on $C$). By considering covariant functors $C\to\mathbf{Sets}$ instead of presheaves, one can also prove a covariant version of the Yoneda lemma, which states that the natural transformations $C(A, -)\Rightarrow F$ (for $F\colon C\to \mathbf{Sets}$ any functor) are in one-to-one correspondence with the elements of $F(A)$. From this one gets an embedding of $C$ into $[C, \mathbf{Sets}]^\text{op}$.
So to sum up, one can embed $C$ into both $[C^\text{op}, \mathbf{Sets}]$ and $[C, \mathbf{Sets}]^\text{op}$. Question: How do $[C^\text{op}, \mathbf{Sets}]$ and $[C, \mathbf{Sets}]^\text{op}$ relate to each other?
What I find a bit weird about the situation: At first, I expected $[C^\text{op}, \mathbf{Sets}]$ and $[C, \mathbf{Sets}]^\text{op}$ to be equivalent. But a general fact is that
$$[C, D]^\text{op}\cong [C^\text{op}, D^\text{op}],$$
thus
$$[C, \mathbf{Sets}]^\text{op}\cong[C^\text{op}, \mathbf{Sets}^\text{op}],$$
which is not $[C^\text{op}, \mathbf{Sets}]$.
Best Answer
Here's one possible answer to this question.
Let's take the viewpoint that functors are representations of categories.
First, why is this sensible?
Well, recall that categories are generalizations of monoids (and consequently groups as well), since a one object category is the same thing as a monoid. If $M$ is a monoid, then we can define a category, $C$, with one object, $*$, hom set $C(*,*)=M$, and unit and composition given by the unit and multiplication in $M$. Conversely, given a one object category $C$, $C(*,*)$ is a monoid with composition as multiplication, and these constructions are inverse to each other.
From now on, if $M$ is a monoid, or $G$ is a group, I'll write $BM$ or $BG$ for the corresponding one object category.
Now, what about functors? Well, what are functors $[BG,k\newcommand\Vect{\text{-}\mathbf{Vect}}\Vect]$?
Well, we need to pick a vector space $V$ to send $*$ to, and we need to pick a monoid homomorphism $G\to \newcommand\End{\operatorname{End}}\End V$. Since $G$ is a group, this is equivalent to a group homomorphism $G\to \operatorname{GL}(V)$. In other words, functors from $BG$ to $k\Vect$ are exactly the same as linear group representations, and you can check that natural transformations of functors correspond exactly to the $G$-equivariant linear maps.
Similarly, when we replace $k\Vect$ with $\newcommand\Ab{\mathbf{Ab}}\Ab$, or $\newcommand\Set{\mathbf{Set}}\Set$, we get $G$-modules and $G$-sets respectively.
Specifically, these are all left $G$-actions, since a functor $F:BG\to \Set$ must preserve composition, so $F(gh)=F(g)F(h)$, and we define $g\cdot x$ by $F(g)(x)$. Thus $(gh)\cdot x = g\cdot (h\cdot x))$.
A contravariant functor $\newcommand\op{\text{op}}BG^\op\to \Set$ gives a right $G$-action, since now $F(gh)=F(h)F(g)$, so if we define $x\cdot g = F(g)(x)$, then we have $$x\cdot (gh) =F(gh)(x) = F(h)F(g)x = F(h)(x\cdot g) = (x\cdot g)\cdot h.$$
Thus we should think of covariant functors $[C,\Set]$ as left $C$-actions in $\Set$, and we should think of contravariant functors $[C^\op,\Set]$ as right $C$-actions in $\Set$.
Yoneda Lemma in Context
Representable presheaves now correspond to free objects in a single variable in the following sense.
The Yoneda lemma is that we have a natural isomorphism $$ [C^\op,\Set](C(-,A),F)\simeq F(A)\simeq \Set(*,F(A)). $$
In other words, $C(-,A)$ looks a lot like the left adjoint to the "forgetful" functor that sends a presheaf $F$ to its evaluation at $A$, $F(A)$, but evaluated on the singleton set $*$.
In fact, we can turn $C(-,A)$ into a full left adjoint by noting that $$\Set(S,F(A)) \simeq \prod_{s\in S} F(A) \simeq \prod_{s\in S}[C^\op,\Set](C(-,A),F) \simeq [C^\op,\Set](\coprod_{s\in S} C(-,A), F),$$ and $\coprod_{s\in S} C(-,A)\simeq S\times C(-,A)$.
Thus one way of stating the Yoneda lemma is that $S\mapsto S\times C(-,A)$ is left adjoint to the evaluation at $A$ functor (in the sense that the two statements are equivalent via a short proof). Incidentally, there is also a right adjoint to the evaluation at $A$ functor, see here for the argument.
Relating this back to more familiar notions
First thing to notice in this viewpoint is that we now have notions of "free on an object" rather than just "free." I.e., I tend to think of $C(-,A)$ as being the free presheaf in one variable on $A$ (this is not standard terminology, just how I think of it).
Now we should be careful, a free object isn't just an object, it's an object and a basis. In this case, our basis (element that freely generates the presheaf) is the identity element $1_A$.
Thinking about it this way, the proof of the Yoneda lemma should hopefully be more intuitive. After all, the proof of the Yoneda lemma is the following:
$C(-,A)$ is generated by $1_A$, since $f^*1_A=f$, for any $f\in C(B,A)$, so natural transformations $C(-,A)$ to $F$ are uniquely determined by where they send $1_A$. (Analogous to saying $1_A$ spans $C(-,A)$). Moreover, any choice $\alpha\in F(A)$ of where to send $1_A$ is valid, since we can define a natural transformation by "extending linearly" $f=f^*1_A \mapsto f^*\alpha$ (this is analogous to saying $1_A$ is linearly independent, or forms a basis).
The covariant version of the Yoneda lemma is the exact same idea, except that we are now working with left representations of our category.
Examples of the Yoneda lemma in more familiar contexts
Consider the one object category $BG$, then the Yoneda lemma says that the right regular representation of $G$ is the free right $G$-set in one variable (with the basis element being the identity, $1_G$). (The free one in $n$-variables is the disjoint union of $n$ copies of the right regular representation.)
The embedding statement is now that $G$ can be embedded into $\operatorname{Sym}(G)$ via $g\mapsto -\cdot g$.
This also works in enriched contexts. A ring is precisely a one object category enriched in abelian groups, and the Yoneda lemma in this context says that the right action of $R$ on itself (often denoted $R_R$) is the free right $R$-module in one variable, with the basis being the unit element $1_R$. (The free one in $n$-variables is now the direct sum of $n$ copies of $R_R$)
The embedding statement here is that $R$ can be embedded into the endomorphism ring of its underlying abelian group via $r\mapsto (-\cdot r)$.