MATLAB: What is wrong for the polynomial regression with gradient descent

polynomial regression gradient descent

i just want to make a polynomial regression with gradient descent ,the funtion i make regression is sin(x),but i am surprised ,if i make regression other function ,just like ax+b ax^2+bx+c,it is succeed,but for sinx,however i change the alpha,it can't succeed,what is problem about my code?
%set the data
N=20
X=(0.1:0.01:2*pi)
Y=sin(X)
r=randi(size(X),[N,1])
X_T=X(r)
Y_T=Y(r)
Y_T=Y_T+randn(1,N)/10
Y_T=Y_T'
hold on;
scatter(X_T,Y_T,'Red'); 
plot(X,Y)
% GRADIENT DESCENT
W_T = randn(1,5);     % initialize W to all zeros
N_W=W_T
V  =[ones(N,1) X_T' (X_T.^2)'  (X_T.^3)' (X_T.^4)']
alpha = 0.00000000005;    % gradient descent step size
W_C=inv(V'*V)*V'*Y_T;
% for each iteration
while 1
for j = 1:5     % looping for each parameter W(i)
    % calculate new value of W(i)
    N_W(j) = W_T(j) -alpha *sum(((V(:,j)*W_T(j)') - Y_T).* V(:,j));
end 
 if(norm(W_T-N_W)<0.001)
     break
 end
 W_T=N_W;
end
T_X=(0.1:0.1:2*pi)
T_Y=N_W(1)+T_X.*N_W(2)+(T_X.^2).*N_W(3)+(T_X.^3).*N_W(4)+(T_X.^4).*N_W(5)
plot(T_X,T_Y, 'Color','Green');
% output actual weights from formula in Red W = [5.775 0.474]'
% finish off
hold off;
[EDITED, Jan, code formatted]

Best Answer

Your stopping criterion isn't a very good one
if(norm(W_T-N_W)<0.001)
   break
end
It is easy to satisfy this criterion because alpha is so small and the algorithm is taking very small steps. A better criterion would be to watch for the gradient norm to get sufficiently small, which could take a long time, because the problem is rather ill-conditioned, cond(V)=~1e4, and because you are taking very small steps.
Related Question