I think is a super basic question but I just can't seem to wrap my head around how think about this.
Let $f: \mathbb{R}^n \rightarrow \mathbb{R}$ be differentiable. We fix an $x \in \mathbb{R}^n$ and define the function $g: \mathbb{R} \rightarrow \mathbb{R}$ by $g(t) = f(tx), \forall t \in \mathbb{R}$.
Now, my question is what is $g'(t)$? If I were deriving $f(tx)$ with respect to $x$, this is easy, I have $f'(tx)t$, but it's not really clear how to derive with respect to $t$.
Is it $f'(tx)x = \nabla f(tx)x$? If yes, why? Wouldn't that imply that $g'(t) \in \mathbb{R}^n$, since $f'(tx) =\nabla f(tx) \in \mathbb{R}$ and $x \in \mathbb{R}^n$?
I apologize if this a super basic question and I'm just missing something crucial!
EDIT: Ok, I think I have an understanding now. Here's what I have:
Let $h: \mathbb{R} \rightarrow \mathbb{R}^n, h(t) = tx$. Now, we have $g(t) = f(h(t))$, and so, by the chain rule:
\begin{align}
Dg(t) & = Df(h(t))\cdot Dh(t) = \begin{pmatrix}\frac{\partial f}{\partial h_1(t)} & \dots & \frac{\partial f}{\partial h_n(t)}\end{pmatrix} \begin{pmatrix}\frac{\partial h_1}{\partial t} \\ \vdots \\ \frac{\partial h_n}{\partial t}\end{pmatrix} \\[10pt]
& = \frac{\partial f}{\partial h_1(t)}\frac{\partial h_1}{\partial t} + \dots + \frac{\partial f}{\partial h_n(t)}\frac{\partial h_n}{\partial t} \\[10pt]
& = \frac{\partial f}{\partial h_1(t)}x_1 + \dots + \frac{\partial f}{\partial h_n(t)}x_n = \frac{\partial f}{\partial tx_1}x_1 + \dots + \frac{\partial f}{\partial tx_n}x_n = Df(tx)x.
\end{align}
Is this understanding correct? Moreover, is $\nabla f(x) = \begin{pmatrix}\frac{\partial f}{\partial x_1} & \dots & \frac{\partial f}{\partial x_n}\end{pmatrix}$ or $\nabla f(x) = \frac{\partial f}{\partial x_1} + \dots + \frac{\partial f}{\partial x_n}$? I thought it was the latter, which led to my confusion, but dealing with everything in jacobian form, it seems that it is the former?
Best Answer
Just for sake of completeness and other users who will ask something similar; Suppose we are given a differentiable function $f: \mathbb{R}^n \to \mathbb{R}$ and $h: \mathbb{R} \to \mathbb{R}^n$ s.t $g \in \textbf{dom}(f)$ and $h$ is also diff'ble. Then we can apply that chain rule,
$$ \frac{d}{dt} (f (h(t)) = \nabla f(h(t)) \cdot h'(t)$$
Here $h(t) = \textbf{x}t$ and so by the above,
\begin{align*} \frac{d}{dt} f(h(t)) &= \nabla f(h(t)) \cdot h'(t) \\ \\ & = \nabla f (\textbf{x}t) \cdot \textbf{x}\end{align*}