First, $\mathbf{F} = x\mathbf{\hat i} + y\mathbf{\hat j} + z\mathbf{\hat k}$ converted to spherical coordinates is just $\mathbf{F} = \rho \boldsymbol{\hat\rho} $. This is because $\mathbf{F}$ is a radially outward-pointing vector field, and so points in the direction of $\boldsymbol{\hat\rho}$, and the vector associated with $(x,y,z)$ has magnitude $|\mathbf{F}(x,y,z)| = \sqrt{x^2+y^2+z^2} = \rho$, the distance from the origin to $(x,y,z)$.
You also asked about where
$$\begin{bmatrix}\boldsymbol{\hat\rho} \\ \boldsymbol{\hat\theta} \\ \boldsymbol{\hat\phi} \end{bmatrix} = \begin{bmatrix} \sin\theta\cos\phi & \sin\theta\sin\phi & \cos\theta \\ \cos\theta\cos\phi & \cos\theta\sin\phi & -\sin\theta \\ -\sin\phi & \cos\phi & 0 \end{bmatrix} \begin{bmatrix} \mathbf{\hat x} \\ \mathbf{\hat y} \\ \mathbf{\hat z} \end{bmatrix}$$
comes from. Let's look at the simpler 2D case first. For a point $(x,y)$ it helps to imagine that you're on a circle centered at the origin. In this case, the two fundamental directions you can move are perpendicular to the circle or along the circle. For the perpendicular direction we use the outward-pointing radial unit vector $\mathbf{\hat{r}}$. For the other direction, moving along the circle means (instantaneously) that you're moving tangent to it, and we take the unit vector in this case to be $\boldsymbol{\hat\theta}$, pointing counterclockwise. For example, suppose you're at the point $(1/\sqrt{2},1/\sqrt{2})$. Then, in the graph below, $\mathbf{\hat{r}}$ is in red and $\boldsymbol{\hat\theta}$ is in yellow.
Note that this means that, unlike the unit vectors in Cartesian coordinates, $\mathbf{\hat{r}}$ and $\boldsymbol{\hat{\theta}}$ aren't constant; they change depending on the value of $(x,y)$.
Now, what about a formula for $\mathbf{\hat{r}}$? If we move perpendicular to the circle we're keeping $\theta$ fixed in the polar coordinate representation $(r \cos \theta, r \sin \theta)$. The vector $\mathbf{\hat{r}}$ is the unit vector in the direction of this motion. If we interpret $r$ as time, taking the derivative with respect to $r$ will give us the velocity vector, which we know points in the direction of motion. Thus we want the unit vector in the direction of $\frac{d}{dr} (r \cos \theta, r \sin \theta) = (\cos \theta, \sin \theta)$. This is already a unit vector, so $\mathbf{\hat{r}} = \cos \theta \mathbf{\hat{x}} + \sin \theta \mathbf{\hat{y}} $. Similarly, moving counterclockwise along the circle entails keeping $r$ fixed in the polar coordinate representation $(r \cos \theta, r \sin \theta)$. Thus to find $\boldsymbol{\hat\theta}$ we take $\frac{d}{d\theta} (r \cos \theta, r \sin \theta) = (-r \sin \theta, r \cos \theta)$. This is not necessarily a unit vector, and so we need to normalize it. Doing so yields $\boldsymbol{\hat\theta}= -\sin \theta \mathbf{\hat{x}} + \cos \theta \mathbf{\hat{y}} $. In matrix form, this is $\begin{bmatrix} \mathbf{\hat{r}} \\ \boldsymbol{\hat\theta}\end{bmatrix} = \begin{bmatrix} \cos \theta & \sin \theta \\ -\sin \theta & \cos \theta \end{bmatrix} \begin{bmatrix} \mathbf{\hat{x}} \\ \mathbf{\hat{y}} \end{bmatrix}$.
Moving up to spherical coordinates, for a given point $(x,y,z)$, imagine that you're on the surface of a sphere. The three fundamental directions are perpendicular to the sphere, along a line of longitude, or along a line of latitude. The first corresponds to $\boldsymbol{\hat\rho}$, the second to $\boldsymbol{\hat\theta}$, and the third to $\boldsymbol{\hat{\phi}}$. (This is using the convention in the Wikipedia page, which has $\theta$ and $\phi$ reversed from what you have.) Thus to find $\boldsymbol{\hat\rho}$, $\boldsymbol{\hat\theta}$, and $\boldsymbol{\hat{\phi}}$, we take the derivative of the spherical coordinate representation $(\rho \sin \theta \cos \phi, \rho \sin \theta \sin \phi, \rho \cos \theta)$ with respect to $\rho$, $\theta$, and $\phi$, respectively, and then normalize each one. That's where the matrix
$$\begin{bmatrix}\boldsymbol{\hat\rho} \\ \boldsymbol{\hat\theta} \\ \boldsymbol{\hat\phi} \end{bmatrix} = \begin{bmatrix} \sin\theta\cos\phi & \sin\theta\sin\phi & \cos\theta \\ \cos\theta\cos\phi & \cos\theta\sin\phi & -\sin\theta \\ -\sin\phi & \cos\phi & 0 \end{bmatrix} \begin{bmatrix} \mathbf{\hat x} \\ \mathbf{\hat y} \\ \mathbf{\hat z} \end{bmatrix}$$
comes from.
As Henning Makholm points out, one way to view what we're doing here is that we're rotating the $\mathbf{\hat{x}}, \mathbf{\hat{y}}, \mathbf{\hat{z}}$ vectors. The transformation matrix can thus be considered a change-of-basis matrix. This means you could also (and more generally) convert $\mathbf{F} = x\mathbf{\hat i} + y\mathbf{\hat j} + z\mathbf{\hat k}$ to spherical coordinates via
\begin{align}
\mathbf{F}
&=
\begin{bmatrix} F_{\rho}\\ F_{\theta}\\ F_{\phi}\end{bmatrix}
\\
&=
\begin{bmatrix} \sin \theta \cos \phi & \sin \theta \sin \phi & \cos \theta \\ \cos \theta \cos \phi & \cos \theta \sin \phi & -
\sin \theta \\ - \sin \phi & \cos \phi & 0 \end{bmatrix} \begin{bmatrix} x \\ y \\ z \end{bmatrix}
\\
&=
\begin{bmatrix} \sin \theta \cos \phi & \sin \theta \sin \phi & \cos \theta \\ \cos \theta \cos \phi & \cos \theta \sin \phi & - \sin \theta \\ - \sin \phi & \cos \phi & 0 \end{bmatrix} \begin{bmatrix} \rho \sin \theta \cos \phi \\ \rho \sin \theta \sin \phi \\ \rho \cos \theta \end{bmatrix}
\\
&=
\begin{bmatrix} \rho \sin^2 \theta \cos^2 \phi + \rho \sin^2 \theta \sin^2 \phi + \rho \cos^2 \theta \\ \rho \sin \theta \cos \theta \cos^2 \phi + \rho \sin \theta \cos \theta \sin^2 \phi - \rho \sin \theta \cos \theta \\ - \rho \sin \theta \sin \phi \cos \phi + \rho \sin \theta \sin \phi \cos \phi + 0 \end{bmatrix}
\\
&= \begin{bmatrix} \rho \\ 0 \\ 0\end{bmatrix}.
\end{align}
So we get $\mathbf{F} = \rho \boldsymbol{\hat\rho} + 0 \boldsymbol{\hat\theta} + 0 \boldsymbol{\hat{\phi}} = \rho \boldsymbol{\hat{\rho}}$, just as we argued at the beginning.
I can partially answer this. I believe your first matrix is not the correct general transformation matrix for cartesian to spherical coordinates because you are missing factors of $\rho$ (the radial coordinate), as well as some other incorrect pieces. So it is not clear what you are trying to show.
If you are trying to derive the general transformation matrix from spherical to cartesian, it is:
$$\begin{bmatrix}
A_x\\
A_y\\
A_z
\end{bmatrix}
= \begin{bmatrix}
\sin \theta \cos \phi & \rho \cos \theta \cos \phi & -\rho \sin \theta \sin\phi\\
\sin \theta \sin \phi & \rho \cos \theta \sin \phi & \rho \sin\theta \cos\phi\\
\cos\theta & -\rho \sin\theta & 0
\end{bmatrix}
\begin{bmatrix}
A_r\\
A_\theta\\
A_\phi
\end{bmatrix}
$$
This matrix is formed from the derivative matrix in the following way:
$
\sum_{(i,j)} \frac {\partial x^i} {\partial u^j}
$
where
$$x^1 = \rho\sin \theta \cos \phi, x^2 = \rho\sin \theta \sin \phi, x^3=\rho\cos\theta$$
and
$u^1 = \rho, u^2 = \theta, u^3 = \phi
$
(Note: the superscripts are indices, not exponents).
If you write out all the derivatives, you get this matrix:
$$
\begin{bmatrix}
A_x\\
A_y\\
A_z
\end{bmatrix}
= \begin{bmatrix}
\frac {\partial (\rho \sin \theta \cos \phi)} {\partial \rho} &
\frac {\partial (\rho \sin \theta \cos \phi)} {\partial \theta} &
\frac {\partial (\rho \sin \theta \cos \phi)} {\partial \phi}\\
\frac {\partial (\rho \sin \theta \sin \phi)} {\partial \rho} &
\frac {\partial (\rho \sin \theta \sin \phi)} {\partial \theta} &
\frac {\partial (\rho \sin \theta \sin \phi)} {\partial \phi}\\
\frac {\partial (\rho \cos \theta)} {\partial \rho} &
\frac {\partial (\rho \cos \theta)} {\partial \theta} &
\frac {\partial (\rho \cos \theta)} {\partial \phi}
\end{bmatrix}
\begin{bmatrix}
A_r\\
A_\theta\\
A_\phi
\end{bmatrix}
$$
...which evaluates to the correct general transformation matrix I listed above.
Best Answer
All right ! See example 1, p. 3 in this pdf.