where do these concepts start to differ?
They are different things by definitions. One might say that they look similar because both of them have "linearity" as a key ingredient in the definitions: for dual spaces, one has linear forms; for inner product spaces, one has bilinear forms.
A well known connection between these two concepts is given by the Riesz representation theorem.
Let $V$ be a real Hilbert space (namely, complete inner product space over $\mathbb{R}$). On the one hand, for any fixed $x\in V$,
$$
y\mapsto \langle x,y\rangle\tag{1}
$$
gives you a (continuous) linear functional on $V$, namely an element in $V^*$. On the other hand, the Riesz representation theorem says that every element in $V^*$ (assuming $V^*$ means the continuous dual) are of the form (1).
Disclaimer: Throughout the discussion, I'll always assume that normed spaces are real or complex (and preferably real).
The existence of some $y$ in $C$ such that $\lVert x-y\rVert=\min_{z\in C} \lVert x-z\rVert$ is a consequence of the fact that closed balls are compact. There must be some closed ball $E(x,R)$ such that $E(x,R)\cap C\ne \emptyset$.
Since $C$ is closed and $E(x,R)$ is compact, $E(x,R)\cap C$ is compact; therefore, there is some $y\in E(x,R)\cap C$ that minimizes the continuous function $\lVert x-\bullet\rVert$ on $E(x,R)\cap C$. Now, it is easy to observe that $\min\left\{\lVert x-z\rVert\,:\, z\in C\right\}=\min\left\{\lVert x-z\rVert\,:\, z\in C\cap E(x,R)\right\}$. For, if $z\notin E(x,R)$, then $\lVert x-z\rVert$ is automatically larger than $R$ and thus of the distance from $x$ of any element of $E(x,R)\cap C$.
So $y$ is indeed a minimizer such as the ones you want. Without additional hypothesis uniqueness won't be guaranteed, basically for the same reason it isn't guaranteed in general. If $V=(\Bbb R^2,\lVert\bullet\rVert_\infty)$ and $C=\{1\}\times [-1,1]$ and $x=(0,0)$, then $d(x,y)=1$ for all $x\in C$.
Following this line of thought, you may want to prove that a normed space is strictly convex if and only if it has the property that for all closed convex sets $C$ and $x\notin C$ there is at most one $y$ such that $\lVert x-y\rVert=\min\{\lVert x-z\rVert\,:\, z\in C\}$.
Best Answer
Not every vector space is an inner product space because not every norm satisfies the parallelogram law. As classic counterexamples, consider the the spaces $\mathbb{R}^n$ under the $1$-norm (aka taxicab norm) and the $\infty$-norm (aka maximum/supremum norm).
If you are given an inner-product space (aka Hilbert space), then there is indeed a strong connection between the dual space and the inner product. This result is known as the Riesz representation theorem. Note, however, that his does not mean that we've "defined the same thing twice". The dual space is the set of all linear mappings to the scalar field, whereas the inner product of an inner product space is a particular (bilinear) map on two vectors to the scalar field.