Basically, can you do everything with the equivalent linear univariate
regression model that you could with the multivariate model?
I believe the answer is no.
If your goal is simply either to estimate the effects (parameters in $\mathbf{B}$) or to further make predictions based on the model, then yes it does not matter to adopt which model formulation between the two.
However, to make statistical inferences especially to perform the classical significance testing, the multivariate formulation seems practically irreplaceable. More specifically let me use the typical data analysis in psychology as an example. The data from $n$ subjects are expressed as
$$ \underset{n \times t}{\mathbf{Y}} = \underset{n \times k}{\mathbf{X}} \hspace{2mm}\underset{k \times t}{\mathbf{B}} + \underset{n \times t}{\mathbf{R}},
$$
where the $k-1$ between-subjects explanatory variables (factor or/and quantitative covariates) are coded as the columns in $\mathbf{X}$ while the $t$ repeated-measures (or within-subject) factor levels are represented as simultaneous variables or the columns in $\mathbf{Y}$.
With the above formulation, any general linear hypothesis can be easily expressed as
$$\mathbf{L} \mathbf{B} \mathbf{M} = \mathbf{C},$$
where $\mathbf{L}$ is composed of the weights among the between-subjects explanatory variables while $\mathbf{L}$ contains the weights among levels of the repeated-measures factors, and $\mathbf{C}$ is a constant matrix, usually $\mathbf{0}$.
The beauty of the multivariate system lies in its separation between the two types of variables, between- and within-subject. It is this separation that allows for the easy formulation for three types of significance testing under the multivariate framework: the classical multivariate testing, repeated-measures multivariate testing, and repeated-measures univariate testing. Furthermore, Mauchly testing for sphericity violation and the corresponding correction methods (Greenhouse-Geisser and Huynh-Feldt) also become natural for univariate testing in the multivariate system. This is exactly how the statistical packages implemented those tests such as car in R, GLM in IBM SPSS Statistics, and REPEATED statement in
PROC GLM of SAS.
I'm not so sure whether the formulation matters in Bayesian data analysis, but I doubt the above testing capability could be formulated and implemented under the univariate platform.
Best Answer
In the setting of classical multivariate linear regression, we have the model:
$$Y = X \beta + \epsilon$$
where $X$ represents the independent variables, $Y$ represents multiple response variables, and $\epsilon$ is an i.i.d. Gaussian noise term. Noise has zero mean, and can be correlated across response variables. The maximum likelihood solution for the weights is equivalent to the least squares solution (regardless of noise correlations) [1][2]:
$$\hat{\beta} = (X^T X)^{-1} X^T Y$$
This is equivalent to independently solving a separate regression problem for each response variable. This can be seen from the fact that the $i$th column of $\hat{\beta}$ (containing weights for the $i$th output variable) can be obtained by multiplying $(X^T X)^{-1} X^T$ by the $i$th column of $Y$ (containing values of the $i$th response variable).
However, multivariate linear regression differs from separately solving individual regression problems because statistical inference procedures account for correlations between the multiple response variables (e.g. see [2],[3],[4]). For example, the noise covariance matrix shows up in sampling distributions, test statistics, and interval estimates.
Another difference emerges if we allow each response variable to have its own set of covariates:
$$Y_i = X_i \beta_i + \epsilon_i$$
where $Y_i$ represents the $i$th response variable, and $X_i$ and $\epsilon_i$ represents its corresponding set of covariates and noise term. As above, the noise terms can be correlated across response variables. In this setting, there exist estimators that are more efficient than least squares, and cannot be reduced to solving separate regression problems for each response variable. For example, see [1].
References