Well, this is true when $X$ is normal and projective over $k$ characteristic $0$ (thanks to Ariyan for mentioning below that I forgot to say this--if you want to this work over arbitrary perfect field $k$ you need to replace $\text{Pic}^0_{X/k}$ with ites reduced subgroup scheme--in characteristic $0$ smoothness is guaranteed), say. First of all it's a somewhat important technical point to note that the Albanese is a constructed associated to pointed varieties. Namely, it's associated to a pair $(X,x)$ where $X$ is a variety and $x\in X(k)$. The claim then is that there is an initial map amongst maps from $(X,x)\to (A,e)$ where $A$ is an abelian variety over $k$ and $e\in A(k)$ its identity section.
The idea is actually somewhat simple: suppose that $(X,x)\to (A,e)$ is a map of pointed varieties. This then induces a map $\text{Pic}_{A/k}\to \text{Pic}_{X/k}$ where, and here's where we use the section, we take our choice of $\text{Pic}_{X/k}$ to mean the group scheme representing
$$T\mapsto \left\{(\mathscr{L},i):\mathscr{L}\in\text{Pic}(X\times_k T)\text{ and }i:x^\ast\mathscr{L}\xrightarrow{\approx}\mathcal{O}_T\right\}$$
and similarly for $A$. The point being that the choice of a section gives us a natural functorial description of the connected component of the Picard scheme (i.e. the functor representing the fppf sheafification of $T\mapsto \text{Pic}(X\times_k T)$).
Now, it then follows that the connected component of the identity $\text{Pic}^0_{A/k}\subseteq\text{Pic}_{A/k}$ maps into the connected component of the idenity $\text{Pic}^0_{X/k}\subseteq\text{Pic}_{X/k}$ and thus we obtain a map $\text{Pic}^0_{A/k}\to \text{Pic}^0_{X/k}$. We then need two facts:
1) The group scheme $\text{Pic}^0_{X/k}$ is an abelian variety (I think this is due to Raynaud?). $\newcommand \Pic{\mathrm{Pic}}$
2) $(\text{Pic}^0_{A/k})^\vee\cong A$--this is almost by definition.
Thus, we obtain a map of abelian varieties $(\text{Pic}^0_{X/k})^\vee\to A$. I leave it for you to unravel that if $(X,x)\to(\text{Pic}^0_{X/k})^\vee$ is the obvious map (what is this?) then the above actually shows that the composition $(X,x)\to ((\text{Pic}^0_{X/k})^\vee,e)\to (A,e)$ is $(X,x)\to (A,e)$. Thus, $\text{Alb}(X,x)=(\text{Pic}^0_{X/k},e)$.
So, you see that this was almost definitionally trivial, that the dual of the Picard scheme is the Albanese, once you know 1) and 2) above. In summary: maps of group varieties give you maps (in the opposite direction) on Picard schemes, and so one gets a map of group varieteis (in the right direction!) on the dual of Picard schemes. Once you know now the Picard schemes look like for general varieties, and once you realize we defined the dual abelian variety in terms of its Picard scheme, everything is perfect.
I think for your question about 'free abelian variety' on $(X,x)$ you mean the 'group law theorem' coming from the Abel-Jacobi map. I have to run now--let me explain that later if no one else has.
EDIT:
Now that I have more time, let me tie up some loose ends. $\newcommand \Alb{\mathrm{Alb}}$
Let me first address this discussion of $\text{Alb}(X,x)$ as being the 'free abelian variety on the points of $X$'. Specifically, let me state the result that says something to this affect, and then interpret it more philosophically:
Theorem: Let $X$ be a variety over $k$ (a perfect field) and $f:(X,x)\to (\Alb(X,x),e)$ an Albanese variety (i.e. an initial abelian variety). Then, for $n\gg 0$ the map $X^n\to \Alb(X,x)$ defined by $\displaystyle (x_1,\ldots,x_n)\mapsto \sum_{i=1}^n f(x_i)$ is a surjection.
What this tells you is that, roughly, you can think of $\text{Alb}(X,x)$ as being something like $X^n/\sim$ (for a suitable equivalence relation $\sim$) with addition being formal addition in $X^n$. In fact, you can see that since $\Alb(X,x)$ is abelian, the map $X^n\to \Alb(X,x)$ factors through $X^{(n)}:=X^n/S_n$.
To give an indication why this is not so surprising from the perspective of Picard varieties, let's restrict our attention for a second to the case when $X=C$ a smooth projective curve over $k$ (algebraically closed, and characteristic $0$ for convenience). Why should something like $C^n$ have anything to do with $\text{Pic}^0_{C/k}$--why should 'free abelian groups on points' have anything to do with Picard groups?
The answer is actually simpler than you might think! Namely, let's imagine what an element of the free abelian group on $C$ is. It's really just a formal linear combination $n_1 p_1+\cdots+n_mp_m$ of points on $C$. But, such objects are also known by another name: divisors. But, divisors are classically known to have a lot to do with line bundles: they're the same thing! Namely, recall that for any divisor $D$ one can associate a line bundle $\mathcal{O}(D)$ and this map from the class group $\text{Cl}(C)$ (divisors modulo principal divisors) to $\text{Pic}(C)$ is an isomorphism.
So, for example what is the map $C^n\to\Alb(X,x)$ if we think of $\Alb(C,x)$ as being $\Pic^0_{C/k}$ (NB: for pointed curves the Picard variety is principally polarized--there is a canonical identification between $\Pic^0_{C/k}$ and its dual--one usually calls this common object the Jacobian variety of $C$)? Well, the naive guess is simple. Namely, to an $n$-tuple $(p_1,\ldots,p_n)$ we get a divisor $p_1+\cdots+p_n$ and thus we get a line bundle $\mathcal{O}(p_1+\cdots+p_n)$. But, this can't be the map $C^n\to\text{Pic}^0_{C/k}$ since $\mathcal{O}(p_1+\cdots+p_n)$ isn't degree $0$--it's degree $n$. That said, this is precisely where our choice of base point becomes pivotal. Namely, we want a canonical way to take $p_1+\cdots+p_n$ and turn it into a degree $0$ divisor, and given our fixed base point $x$ the way is clear: $p_1+\cdots+p_n-nx$. Thus, our map $C^n\to\text{Pic}^0_{C/k}$ is $(p_1,\ldots,p_n)\mapsto \mathcal{O}(p_1+\cdots+p_n-nx)$.
So the above Theorem then says something to the effect that every degree $0$ line bundle looks like $\mathcal{O}(p_1+\cdots+p_n-nx)$ for points $(p_1,\ldots,p_n)\in C^n$ if $n\gg 0$.
So, to summarize: what is the content of the relationship between the 'free abelian variety on the points of $X$' perspective and the 'Picard variety perspective'? It's that the Picard group is the same thing as the class group!
Best Answer
$\mathrm{Pic}^0$ is meaningful only for complete varieties. If the variety is not normal, this is defined as an abelian group, but not as a scheme (see Variete du Picard, by CS Seshadri). To define as a group, one can do the following. Line bundles can be defined for any scheme the usual way, no mention of divisors necessary. For any non-singular projective curve $C$ and a morphism $f:C\to V$, one has the pull back map $\mathrm{Pic}\, V\to \mathrm{Pic}\, C$ and thus the inverse image of $\mathrm{Pic}^0C$ in $\mathrm{Pic}\, V$ (Notice that $\mathrm{Pic}^0C$ makes sense since $C$ is a non-singular projective curve). The intersection of all these subgroups as $C$ (and $f$) varies is called $\mathrm{Pic}^0 V$.