[Math] Vector orthogonal to hyperplane

Question

Finding the vector perpendicular to the plane

Why is weight vector orthogonal to decision plane in neural networks

I am thinking now about hyperplanes and orthogonal vectors. My problem is following one:

Definition of hyperplane:
$$H=\left\{ x\ \epsilon\ R^2 : u^Tx=v \right\}$$

Consider for instance simple case of hyperplane in $R^2$ (straight line):

$$H=\left\{ x\ \epsilon\ R^2 : u^Tx=3 \right\}$$

so that $x_{1}-x_{2}=3$ so $u=\left(\begin{array}{c}1\\ -1\end{array}\right)$

take any point from that straight line : $x=\left(\begin{array}{c}4\\ 1\end{array}\right)$
$$u^Tx\neq0$$

So we conclude that points on this line (hyperplane) are not orhogonal to vector of coefficients $u$, and this is the case (they are othogonal) only if:
$$H=\left\{ x\ \epsilon\ R^2 : u^Tx=0 \right\}$$

In case of plane we have following definition of hyperplane :
$$H=\left\{ x\ \epsilon\ R^3 : u^Tx=v \right\}$$

then similarily $x$ and $u$ are orthogonal only in case $v=0$

I came across following definition which confused me a little bit:

$$H=\left\{ x\ \epsilon\ R^n : u^Tx=v \right\}=\left\{ x\ \epsilon\ R^n : u^T(x-a)=0 \right\}$$ where $a$ is any point on hyperplane ( so $u^Ta-v=0$). Therefore, the hyperplane H consists of the points x for which $<u, x — a> = 0$.
In other words, the hyperplane H consists of the points x for which the vectors u and x-a are orthogonal

So this is the case such that $(x-a)\ \epsilon\ R^n$ and $v=0$, thats why $(x-a)$ is orthogonal to $u$.

So why in literature about for instance about Suppor Vector Machines, we assume that vector of coefficients $u$ is orthogonal to separating hyperplane, for cases where $v\neq0$? Shouldn't it be like if $u$ is orthogonal to some hyperplane, then $u$ is orthogonal to every vector $x$ in such hyperplane?

I certainly misuse some concepts, so could you provide step by step proof of the fact that vector of coefficients $u$ is orthogonal to any hyperplane?

Answer 1

So to formalize discussion with Triatticus, proof is following:

If we have some hyperplane given by: $$H=\left\{ x\ \epsilon\ R^n : u^Tx=v \right\}$$

Then any two vectors $a,b$ which point to any separate points in this hyperplane are such that:

$$\begin{cases}u^Ta=v \\u^Tb=v \end{cases}$$

If we want to check whether hyperplane is orthogonal to normal vector $u$, then we have to find whether $u^T(a-b)=0$ for any a,b , because $(a-b)$ is a vector IN the hyperplane, whereas a and b are vectors which point in direction of points in the hyperplane.

If we solve above linear equations we have : $$u^T(a-b)=0$$ and this completes proof.