Solved – the relationship between orthogonality and the expectation of the product of RVs

conditional-expectationlinear algebraregression

Is there such thing as a statistical concept of orthogonality? Does somebody could provide a formal explanation about the relationship between orthogonality and conditional expectation of a RV? Here is the motivation for the question. In Greene (2011) Econometric Analysis, pg. 93, he writes

(1) "Assumption A3 states that the disturbances in the population are stochastically orthogonal to the independent variables in the model; that is, $E[\epsilon|\vec{x}]=0$"

A3: $E[\epsilon|\vec{x}]=E[\epsilon|x_1,x_2,…,x_n]=0$

Why $E[\epsilon|\vec{x}]=0$ is the same thing as to say that $e_i$ is stochastically orthogonal of each $x_i$, the independent variables in the model. How to random variables can be orthogonal?

Definition of orthogonality: it is defined in Linear Algebra and it requires at least two vectors. One possible way to say two vectors are orthogonal is that their dot product is zero, that is, if $x=(x_1,…,x_n)$ and $y=(y_1,…,y_n)$ then

$x\cdot y = 0$

Definition of conditional expectation:

$E[\epsilon|\vec{x}] = \int_\epsilon \epsilon f(\epsilon|\vec{x})d\epsilon$

How the two concepts are formally related?

Best Answer

Notice that orthogonality is concept that derives its self intuitions about vectors in $R^n$ were we say if dot product between two vectors is 0 then they are orthogonal. Well,in linear algebra this idea is generalized function and the dot prodcut is replaced by a general called an inner product, $\langle x,y\rangle$, (which is basically a function, letting $V$ be a vector space, $ V^2\rightarrow \mathbb{R}$ that follows certain criteria). And here we say vectors are orthogonal if their inner product of the vectors is 0.

Thus all you need to have idea of orthogonality with RVs as your vectors, is to define an inner product. Well, the usual inner product used and one that fulfills the criteria for an inner product is $$\langle X,Y \rangle= E(XY)$$ Thus RVs, X and Y are orthogonal if $E(XY)=0$

Related Question