Suppose $X_1,X_2$ are independent standard normal distributions, and suppose $\alpha = (X_1, X_1 \cos\theta + X_2 \sin\theta), \beta = ( X_1 \cos\theta + X_2 \sin\theta, X_1)$.
Then both $\alpha$ and $\beta$ are bivariate normal distributions and they have the same means and covariance matrices. But for some $\theta$, e.g. $\theta = 0.1$, it would seem like they have different distributions. More formally, we can see that the joint density $f_{\beta} = f_{\alpha} \circ R$, where $R = \begin{bmatrix}
0 & 1\\
1 & 0
\end{bmatrix}$.
So how does it make sense that multivariate normal distributions are parametrized by means and covariance? Do we always assume that if the means are the same in all components, then the order of the variables is in some sense ambiguous?
EDIT:
This example seems to show that for any bivariate (and for multivariate too) distribution with constant mean in its coordinates, we have that switching variables preserves the covariance matrix. Unless the distribution is symmetric with respect to $R$ mentioned above, switching coordinates will give a different resulting distribution. So parametrizing multivariate distributions by means and covariances seems problematic in general.
Best Answer
$Y_1=X_1$ and $Y_2=X_1\cos\theta+X_2\sin\theta$ in your case are an example of exchangeable random variables, i.e., $(Y_1,Y_2)\overset{d}{=}(Y_2,Y_1)$. In general, a normal random vector $Y\sim \mathcal{N}(\mu,\Sigma)$ in $\mathbb{R}^d$ is exchangeable iff $$ \mu=P_{\pi}\mu\quad\text{and}\quad \Sigma=P_{\pi}\Sigma P_{\pi}^{\top} $$ for any permutation $\pi$ of $\{1,\ldots, d\}$ ($P_{\pi}$ here is the corresponding permutation matrix).