In follow,
with the star symbol, I mean complex conjugate, i.e. for example
$(3-2i,3,2i)^*=(3+2i,3,-2i)$,
If $v=\{{v_1,v_2,v_3}\}$ and $u=\{{u_1,u_2,u_3}\}$ , we define
$v.u=v_1u_1+v_2 u_2+v_3 u_3$ and $<v,u>=u^*.v$.
The detailed answer is as follow,
1.
$v_1=(1,i,0)$ and $v_2=(1-i,2,4i)$, using the Gram Schmidt process, we have
$$w_1=v_1=(1,i,0)$$
$$w_2=v_2-\frac{v_1^*.v_2}{v_1^*.v_1}v_1=(1-i,2,4i)-\frac{(1,-i,0).(1-i,2,4i)}{(1,-i,0).(1,i,0)}(1,i,0)\\ =(1-i,2,4i)-\frac{1-3i}{2}(1,i,0)=(\frac{1+i}{2},\frac{1-i}{2},4i)$$
2.
$$u_1=\frac{{w}_1}{\sqrt{{w}_1^*.{w}_1}}=\frac{(1,i,0)}{\sqrt{(1,-i,0).(1,i,0)}}=\frac{1}{\sqrt{2}}(1,i,0)$$
$$u_2=\frac{{w}_2}{\sqrt{{w}_2^*.{w}_2}}=\frac{(\frac{1+i}{2},\frac{1-i}{2},4i)}{\sqrt{(\frac{1-i}{2},\frac{1+i}{2},-4i).(\frac{1+i}{2},\frac{1-i}{2},4i)}}\\=\frac{1}{\sqrt{17}}(\frac{1+i}{2},\frac{1-i}{2},4i)=\frac{1}{2\sqrt{17}}(1+i,1-i,8i)$$
3.
Since $x \in {C^3}$, we need three orthonormal basis $\{{u_1,u_2,u_3}\}$. The unit vector $u_3$ must be orthogonal to both $u_1$ and $u_2$, therefore, if we assume $u_3=(\alpha_1,\alpha_2,\alpha_3)$ then,
$u_1^*.u_3=0 \to \frac{1}{\sqrt{2}}(1,-i,0).(\alpha_1,\alpha_2,\alpha_3)=0 \to \alpha_1-i\alpha_2=0 \,\,(1)$
$u_2^*.u_3=0 \to \frac{1}{2\sqrt{17}}(1-i,1+i,-8i).(\alpha_1,\alpha_2,\alpha_3)=0 \to (1-i)\alpha_1+(1+i)\alpha_2-8i\alpha_3=0 \,\,(2)$
Equations $(1)$ and $(2)$ have solutions of the form of $\alpha (4i,4,1 - i)$. Using the normalization condition,
$$\sqrt {{{\left| {{\alpha _1}} \right|}^2} + {{\left| {{\alpha _2}} \right|}^2} + {{\left| {{\alpha _3}} \right|}^2}} = 1$$
we obtain,
$$u_3=\frac{1}{\sqrt{34}}(4i,4,1-i)$$
Now, we turn into calculation of Fourier coefficients. From vector equation $$a_1u_1+a_2u_2+a_3u_3=x$$
or
$$\frac{a_1}{\sqrt{2}}(1,i,0)+\frac{a_2}{2\sqrt{17}}(1+i,1-i,8i)+\frac{a_3}{\sqrt{34}}(4i,4,1-i)=(3+i,4i,-4)$$
we have these linear system of equations,
$$\frac{a_1}{\sqrt{2}}+\frac{a_2}{2\sqrt{17}}(1+i)+\frac{4ia_3}{\sqrt{34}}=3+i$$
$$\frac{i a_1}{\sqrt{2}}+\frac{a_2}{2\sqrt{17}}(1-i)+\frac{4a_3}{\sqrt{34}}=4i$$
$$\frac{8ia_2}{2\sqrt{17}}+\frac{a_3}{\sqrt{34}}(1-i)=-4$$
which can be solved easily,
$$a_1=\frac{7+i}{\sqrt{2}}$$
$$a_2=\sqrt{17}i$$
$$a_3=0$$
4.
Using given equation, we have
$$a_1=u_1^*.x=\frac{1}{\sqrt{2}}(1,-i,0).(3+i,4i,-4)=\frac{7+i}{\sqrt{2}}$$
$$a_2=u_2^*.x=\frac{1}{2\sqrt{17}}(1-i,1+i,-8i).(3+i,4i,-4)=\sqrt{17}i$$
$$a_3=u_3^*.x=\frac{1}{\sqrt{34}}(-4i,4,1+i).(3+i,4i,-4)=0$$
Best Answer
The standard method of orthogonalisation works like this:
1) take a basis $\{a_n\}$
2) take first vector $a_1$ , normalise it, call it the first vector in the new basis: $$b_1:=\frac{a_1}{\sqrt{\langle a_1,a_1\rangle}}.$$
3) take second vector $a_2$, find its component orthogonal to the first vector in the new basis: $$a'_2=a_2- \langle a_2,b_1\rangle b_1 ,$$ normalise it, then call it the second vector in the new basis: $$b_2:=\frac{a'_2}{\sqrt{\langle a'_2,a'_2\rangle}}.$$
4) repeat for all consecutive vectors - orthogonalisation comes for all previous vectors in new basis: $$a_k'=a_k- \sum_{j=1}^{k-1} \langle a_k,b_j\rangle b_j,\quad b_k= \frac{a_k'}{\sqrt{\langle a_k',a_k'\rangle}}.$$
So, let's take a canonical basis in $P_2[x]$: $\{1,x,x^2\}$.
First vector is $1$. It's norm with respect to our inner product is $2\int_0^11\cdot 1dx =2$, hence the first vector in the new basis is $b_1=\frac{1}{\sqrt 2}$.
Second vector is $x$. The orthogonalisation yields $$x- \langle x,1/\sqrt 2\rangle 1/\sqrt 2= x-\frac{1}{\sqrt 2}\cdot2\int_0^1\frac{x}{\sqrt 2}dx =x-\frac 12.$$
Can you now normalise it (i.e. find the second vector in the new basis) and then find the third vector in the new basis?