Basically, can you do everything with the equivalent linear univariate
regression model that you could with the multivariate model?
I believe the answer is no.
If your goal is simply either to estimate the effects (parameters in $\mathbf{B}$) or to further make predictions based on the model, then yes it does not matter to adopt which model formulation between the two.
However, to make statistical inferences especially to perform the classical significance testing, the multivariate formulation seems practically irreplaceable. More specifically let me use the typical data analysis in psychology as an example. The data from $n$ subjects are expressed as
$$ \underset{n \times t}{\mathbf{Y}} = \underset{n \times k}{\mathbf{X}} \hspace{2mm}\underset{k \times t}{\mathbf{B}} + \underset{n \times t}{\mathbf{R}},
$$
where the $k-1$ between-subjects explanatory variables (factor or/and quantitative covariates) are coded as the columns in $\mathbf{X}$ while the $t$ repeated-measures (or within-subject) factor levels are represented as simultaneous variables or the columns in $\mathbf{Y}$.
With the above formulation, any general linear hypothesis can be easily expressed as
$$\mathbf{L} \mathbf{B} \mathbf{M} = \mathbf{C},$$
where $\mathbf{L}$ is composed of the weights among the between-subjects explanatory variables while $\mathbf{L}$ contains the weights among levels of the repeated-measures factors, and $\mathbf{C}$ is a constant matrix, usually $\mathbf{0}$.
The beauty of the multivariate system lies in its separation between the two types of variables, between- and within-subject. It is this separation that allows for the easy formulation for three types of significance testing under the multivariate framework: the classical multivariate testing, repeated-measures multivariate testing, and repeated-measures univariate testing. Furthermore, Mauchly testing for sphericity violation and the corresponding correction methods (Greenhouse-Geisser and Huynh-Feldt) also become natural for univariate testing in the multivariate system. This is exactly how the statistical packages implemented those tests such as car in R, GLM in IBM SPSS Statistics, and REPEATED statement in
PROC GLM of SAS.
I'm not so sure whether the formulation matters in Bayesian data analysis, but I doubt the above testing capability could be formulated and implemented under the univariate platform.
As for Question 1, you are correct with what you said.
As for Question 2, multivariate stands for an analysis involving more than one response variables. To my knowledge there is no differentiation in terminology with respect to the predictor variables. To be consistent one could maybe say, but I am not sure, "simple multivariate regression" when multiple responses and one predictor variable are present.
As for Question 3, I'd say you are right again.
As for Question 4, the term bivariate refers to a situation when there are two continuous variables in total, i.e. an analysis that can be visualized in a 2d scatter plot (simple linear regression and correlation for example).
So now what does univariate refers too? I think (and I might be wrong) that is the case when you have one response and one or more categorical predictor(s). So, for example you measure the heights of trees coming from the same parent tree, or the weight of chicken fed with the different feeds. This type of analyses would be analyzed as a t-test or Analysis of Variance.
The difference between univariate and bivariate can be seen when you visualize the data. If you plot something as a bar graph, (or dot plot) it is univariate, if you plot something on a 2d scatter plot, it is bivariate. I might be wrong here but I am sure if that's the case someone will comment!
Best Answer
Be sure to read the full example on the UCLA site that you linked.
Regarding 1:
Using a multivariate model helps you (formally, inferentially) compare coefficients across outcomes.
In that linked example, they use the multivariate model to test whether the
writecoefficient is significantly different for thelocus_of_controloutcome vs for theself_conceptoutcome. I'm no psychologist, but presumably it's interesting to ask whether your writing ability affects/predicts two different psych variables in the same way. (Or, if we don't believe the null, it's still interesting to ask whether you have collected enough data to demonstrate convincingly that the effects really do differ.)If you ran separate univariate analyses, it would be harder to compare the
writecoefficient across the two models. Both estimates would come from the same dataset, so they would be correlated. The multivariate model accounts for this correlation.Also, regarding 4:
There are some very commonly-used multivariate models, such as Repeated Measures ANOVA . With an appropriate study design, imagine that you give each of several drugs to every patient, and measure each patient's health after every drug. Or imagine you measure the same outcome over time, as with longitudinal data, say children's heights over time. Then you have multiple outcomes for each unit (even when they're just repeats of "the same" type of measurement). You'll probably want to do at least some simple contrasts: comparing the effects of drug A vs drug B, or the average effects of drugs A and B vs placebo. For this, Repeated Measures ANOVA is an appropriate multivariate statistical model/analysis.