NOTE: Einstein summation is assumed. Also, the component expanded vector in the standard basis will be expressed with square brackets and in the curvilinear basis with round brackets. E.g, in the case of plane polar coordinates,
$$\begin{bmatrix}
1\\
\sqrt{3}
\end{bmatrix} =\begin{pmatrix}
2\\
\pi /3
\end{pmatrix}$$
Links to similar questions: 1 and 2. The reason I'm asking this question is because the answers given in these posts were pretty unsatisfactory – I don't want to assume the coordinates are orthogonal and I'm ok with the result being somewhat circular in nature. All I'm looking for is a nice succinct formula that gives correct results.
I'm currently studying differential geometry. I'm sure we are all aware of the formula for the inner product of two vectors in the standard basis
$$\langle\underline{u},\underline{v}\rangle_{\text{std.}}=\delta_{ij}u^iv^j$$
However, this is only true if the vectors are expressed in the standard basis. For instance we know that in plane polar coordinates, the dot product is obviously not $r r'+\theta\theta'$. Wikipedia cites the formula
$$\langle\underline{u},\underline{v}\rangle_{\mathscr{C}}=g_{ij}u^iv^j$$
As the inner product for a general coordinate system $\mathscr{C}$.
Now obviously, it shouldn't matter what coordinate basis we take the inner product with respect to. The only reason for the above notation is to emphasize that the form of the inner product will change from one coordinate system to another, but indeed for a particular pair of vectors its value will not.
However when this formula is applied to plane polar coordinates, this gives the strange formula
$$\langle\underline{u},\underline{v}\rangle_{\text{polar}}=u_rv_r+r^2u_\theta v_\theta$$
Which doesn't really make sense. In this context, what even is $r$? In fact, the above formula fails for obvious reasons – the metric tensor varies from point to point, e.g for the vector $[~1~~~1~]^{\mathrm{T}}=(~\sqrt{2}~~~\pi/4~)^{\mathrm{T}}$ the metric tensor is
$$\mathbf{g}(\sqrt{2},\pi/4)=\begin{bmatrix}
\begin{bmatrix}
1 & 0
\end{bmatrix} & \begin{bmatrix}
0 & 2
\end{bmatrix}
\end{bmatrix}$$
Whereas for the vector $[~5~~~12~]^{\mathrm{T}}=(~13~~~\arctan(12/5)~)^{\mathrm{T}}$ it is
$$\mathbf{g}(13,\arctan(12/5))=\begin{bmatrix}
\begin{bmatrix}
1 & 0
\end{bmatrix} & \begin{bmatrix}
0 & 169
\end{bmatrix}
\end{bmatrix}$$
So what are Wiki referring to by $g_{ij}$ in the referenced formula?
But, even if the two vectors have the same metric tensor, i.e, same $r$ component, the formula still fails, as we presumably get
$$\langle(r,\theta_1),(r,\theta_2)\rangle_{\text{polar}}=r^2(1+\theta_1\theta_2)$$
Which is clearly not correct.
So here is my question: Let's suppose we have two vectors in $\mathbb{R}^n$ which in the standard basis are
$$\underline{v}=\begin{bmatrix}
x^{1}\\
\vdots \\
x^{n}
\end{bmatrix} \ ;\ \underline{v'} =\begin{bmatrix}
{x'} ^{1}\\
\vdots \\
{x'} ^{n}
\end{bmatrix}$$
And in some curvilinear basis are
$$\underline{v}=\begin{pmatrix}
q^{1}\\
\vdots \\
q^{n}
\end{pmatrix}~;~\underline{v'}=\begin{pmatrix}
{q'}^{1}\\
\vdots \\
{q'}^{n}
\end{pmatrix}$$
Let's suppose we know the coordinate transformation functions $q^i=q^i(x^1,…,x^n)$ and also the reverse coordinate transformation functions $x^i=x^i(q^1,…,q^n)$ and we are given the explicit form of the metric tensor $\mathbf{g}$ for the coordinate transformation $[~x^1~~~…~~~x^n~]^{\mathrm{T}}\to(~q^1~~~…~~~q^n~)^{\mathrm{T}}$. Given the two vectors in their curvilinear components, is there any way to work out $\langle\underline{v},\underline{v'}\rangle$ without resorting to converting back to the standard basis? That is, is there any magic formula
$$\langle\underline{v},\underline{v'}\rangle_{\mathscr{C}}=f\left((q^1,…,q^n),({q'}^1,…,{q'}^n)\right)$$
Best Answer
Summary: The magic formula you seek really is that $g(v,w)=g_{ij}v^iw^j$, provided you understand the definition of every symbol appearing in this formula.
Seeing as I wrote the answer in the first link, it seems fitting I should say a few words (which unfortunately aren't as few as I'd hoped). In general if you have an $n$-dimensional (pseudo) Riemannian manifold $(M,g)$, then for each $p\in M$, by definition, $g_p$ is an (pseudo)inner product on the tangent space $T_pM$. So, if you have two vectors $v,w\in T_pM$, the quantity $g_p(v,w)$ is an element of $\Bbb{R}$. Now, if you have a chart $(U,x)$ on the manifold (i.e a coordinate system), then for each $p\in U$, we can consider the chart-induced basis $\left\{\frac{\partial}{\partial x^i}(p)\right\}_{i=1}^n$ of $T_pM$. So, if $v,w\in T_pM$ we can write these vectors as linear combinations of the basis vectors as \begin{align} v&= v^i\frac{\partial}{\partial x^i}(p) \quad \text{and} \quad w= w^j\frac{\partial}{\partial x^j}(p) \end{align} for some $v^1,\dots v^n,w^1,\dots, w^n\in\Bbb{R}$ (in fact $v^i= dx^i(p)[v]$; i.e the evaluation of the covector $dx^i(p)\in T_p^*M$ on the vector $v\in T_pM$). So, now if we want to calculate their inner product, we have by bilinearity, \begin{align} g_p(v,w)&=g_p\left(v^i\frac{\partial}{\partial x^i}(p), w^j\frac{\partial}{\partial x^j}(p)\right)\\ &= g_p\left(\frac{\partial}{\partial x^i}(p),\frac{\partial}{\partial x^j}(p)\right)\cdot v^i w^j \end{align} This is clearly alot to write, so we write the last line as $g_{(x)\, ij}(p)$ to indicate that it's the $i,j$ component of $g$ at the point $p$, taken relative to the chart $(U,x)$. It is common to omit the $(x)$, and write this equation as $g_p(v,w)=g_{ij}(p)v^iw^j$, with the understanding that $v=v^i\frac{\partial}{\partial x^i}(p)$ and similarly for $w$.
Also, we usually understand that the point $p$ is always present hence we write this succinctly as $g(v,w)=g_{ij}v^iv^j$; but seeing as your confusion is essentially arising from not carefully distinguishing points of tbe manifold from vectors in the tangent space at that point, I explicitly wrote it all out.
By the way, even if we take $M=\Bbb{R}^n$ to be a trivial manifold you should still be careful to distinguish between points of $M=\Bbb{R}^n$ and vectors in $T_pM=T_p\Bbb{R}^n$. You're right that there is indeed a nice isomorphism $T_p\Bbb{R}^n\cong \Bbb{R}^n$ (where now we think of $\Bbb{R}^n$ as a vector space, not a manifold), but we only get this isomorphism after a choice of chart: the obvious one is the global chart $(\Bbb{R}^n,\text{id}_{\Bbb{R}^n})$. But of course, if you want to use a different chart, then you'll get a different isomorphism. So, blurring the distinction between points of the manifold and vectors in the tangent space (I'd say especially in simple cases like $\Bbb{R}^n$, when you're first learning these things) is never a good idea unless you already know what you're doing (in which case you can of course do whatever you want), or unless you're using the identity chart (i.e cartesian coordinates), because in this case everything will work out nicely though you may not fully appreciate why. So, if you plan on using polar/parabolic/hyperbolic/elliptic/spherical or any other common coordinate systems, then you must be especially careful regarding the distinction (since the $g_{ij}$'s are no longer constant functions).
Let's work through an example to see how the calculations work. Let $M=\Bbb{R}^2$ which we think of as a manifold. Your very first equality $\begin{bmatrix} 1\\ \sqrt{3} \end{bmatrix} =\begin{pmatrix} 2\\ \pi /3 \end{pmatrix}$ is already riddled with confusion, because you write
From this it seems that you think this is an equality of vectors in some tangent space of $\Bbb{R}^2$, when in fact it is not. What you mean is you're considering the manifold $M=\Bbb{R}^2$, on which you have the polar coordinate chart $(r,\theta)$ defined on the open set $U:= \Bbb{R}^2\setminus [0,\infty)\times\{0\}$. Note that $r,\theta$ are functions on $U$, i.e $r:U\to \Bbb{R}$ is defined as $r(a,b):= \sqrt{a^2+b^2}$, and $\theta$ is of course given by a piecewise formula which if restricted to the positive quadrant is just $\theta(a,b)= \arctan\left(\frac{b}{a}\right)$.
Here, when I write $(a,b)$, don't think of this as "a vector in the standard/cartesian basis" or whatever. What I mean is $U$ is a (open) subset of $\Bbb{R}^2$, and $\Bbb{R}^2$ is by definition a set whose elements are tuples of real numbers. That's it. So $(a,b)$ is just a particular element (point) of the set $U$, and $r(a,b)$ is the value of the function $r$ on the element $(a,b)\in U=\text{domain}(r)$. So, your very first equality actually means that if you let $p=(1,\sqrt{3})\in U$, then $r(p)=2$ and $\theta(p)=\frac{\pi}{3}$. In particular it DOES NOT represent the relationship between components of a vector in the tangent space.
So, now, let's fix the point $p=(1,\sqrt{3})$ and consider the vectors $v,w\in T_pM=T_p\Bbb{R}^2$ given by \begin{align} v&= \frac{\partial}{\partial r}(p)+ \frac{\partial}{\partial \theta}(p) \quad \text{and} \quad w=\frac{\partial}{\partial \theta}(p) \end{align} (i.e in the language of what I wrote above, these are the basis vectors of $T_pM$ induced by the polar coordinate chart). i.e if you write $v=v^r\frac{\partial}{\partial r}(p)+v^{\theta}\frac{\partial}{\partial \theta}(p)$, then it should be clear that I'm saying $v^r=v^{\theta}=1$, and likewise that $w^r=0,w^{\theta}=1$. In polar coordinates, the components of the metric tensor are $g_{\text{polar},11}(p)=1$, $g_{\text{polar},22}(p)=[r(p)]^2$ and $g_{\text{polar},12}(p)=g_{\text{polar}21}(p)=0$ i.e we just write $[g]_{\text{polar}}=\text{diag}(1,r^2)$ more succinctly. Then, using the above formula, we have \begin{align} g_p(v,w)&= g_{\text{polar}11}(p)v^rw^r + g_{\text{polar}22}(p)v^{\theta}w^{\theta}\\ &= 1\cdot 1\cdot 0 + [r(p)]^2\cdot 1\cdot 1\\ &= [r(p)]^2\\ &= 4. \end{align}
Just for fun, let's see how things work out if we express $v,w$ in terms of $\frac{\partial}{\partial x}(p)$ and $\frac{\partial}{\partial y}(p)$. So, we have the following "change of basis formula": \begin{align} \begin{cases} \frac{\partial}{\partial r}(p)&= \frac{\partial x}{\partial r}(p)\cdot \frac{\partial}{\partial x}(p) + \frac{\partial y}{\partial r}(p)\cdot \frac{\partial}{\partial y}(p)\\ &= \cos(\theta(p))\frac{\partial}{\partial x}(p)+ \sin(\theta(p))\frac{\partial}{\partial y}(p)\\ &= \frac{1}{2}\frac{\partial}{\partial x}(p) + \frac{\sqrt{3}}{2}\frac{\partial}{\partial y}(p)\\\\ \frac{\partial}{\partial \theta}(p)&= \frac{\partial x}{\partial \theta}(p)\cdot \frac{\partial}{\partial x}(p) + \frac{\partial y}{\partial \theta}(p)\cdot \frac{\partial}{\partial y}(p)\\ &= -r(p)\sin(\theta(p))\frac{\partial}{\partial x}(p) + r(p)\cos(\theta(p))\frac{\partial}{\partial y}(p)\\ &= -\sqrt{3}\frac{\partial}{\partial x}(p) + \frac{\partial}{\partial y}(p) \end{cases} \end{align} So, if you do the substitutions, you should find \begin{align} v&= \left(\frac{1}{2}-\sqrt{3}\right)\frac{\partial}{\partial x}(p) +\left(\frac{\sqrt{3}}{2}+1\right)\frac{\partial}{\partial y}(p) \end{align} and \begin{align} w&= -\sqrt{3}\frac{\partial}{\partial x}(p) + \frac{\partial}{\partial y}(p) \end{align} In other words, we have $v^x=\frac{1}{2}-\sqrt{3}, v^y=\frac{\sqrt{3}}{2}+1, w^x=-\sqrt{3}, w^y=1$. Now, since in cartesian coordinates, the components of the metric are just $[g]_{\text{cart}}=\text{diag}(1,1)$ (almost by definition), we have \begin{align} g_p(v,w)&= v^xw^x + v^yw^y = 4, \end{align} which is precisely what we found above.
I hope this shows you the importance of being careful about what the point $p$ of the manifold is and what the vectors $v,w$ in the tangnet space are, and why it is important not to conflate the two (and also how to do the change of basis within the tangent space).