MATLAB: Stochastic Gradient Descent (SGD) for Image Processing

stochastic gradient descent image processing denoise

Dear all,
I am trying to apply SGD to solve a classical image processing problem as in this link . I am not sure what should I change. Here is the Gradient Descent Code:
niter = 500;    % number of iterations
x = u;          % initial value for x, u is the input noisy image
for i=1:niter
    % smoothed total variation of the image
    gdx = grad(x).^2;
    sgdx=gdx(:,:,1)+gdx(:,:,2);
    NormEps   = sqrt( epsilon^2 + sgdx );
    J         = sum(NormEps(:)) ; % this is a scalar value
    % functional to minimize, lambda is weight of J
    nm=sum((x(:)-u(:)).^2);
    f         = 1/2 * nm^2 + lambda * J;
    % normalized gradient of J
    GradJ     =-div( grad(x)./repmat(NormEps, [1 1 2]) );
    % Gradient Descent update equation
    % the gradient of the functional function f is:
    %  x - y + lambda *    GradJ
    x = x - tau * ( x - u + lambda * GradJ);
end
clf;
imageplot(clamp(x)); % this is the result denoised image
I understand that in SGD we took only random part of the image at each iteration then we compute the minimum, but if I apply this on the input noisy image, I will denoise (badly) small part of the image at each iteration, right? an explanation based on the code above would be excellent!
Best regards,
Ibraheem

Best Answer

Update a random part of the image at each iteration is not SGD. In SGD, the parameter, say x, you want to optimize for all iterations is the same x, but the gradient used to update x is noisy due to replacing expectation with sample average. I checked your image denoising problem. It is a standard convex optimization, and there are many efficient solvers. You left a comment on my psgd post, and I showed how to use psgd to converge to better solutions faster.
Related Question