MATLAB: Circshift slower on GPU

circshiftgpuarray

Unlike other matrix operations, it seems that circshift runs slower on GPU than on CPU.
Run the following code, GPU spends twice the time of CPU. Any method to improve it?
M = rand(512);
N = gpuArray(rand(512));
tic;
for i = 1:10
    circshift(M, [10, 10]);
end
toc;
tic;
for i = 1:10
    circshift(N, [10, 10]);
end
toc;

Best Answer

Yes, circshift is a bit slow on the GPU particularly when it has to shift both rowwise and columnwise, because that means it has to stride the entire array reading and writing out to new locations - the GPU isn't especially good at that in comparison to main memory. You can see the GPU performance overtake the CPU somewhere around arrays of size 1000x1000. On my Kepler card:
>> A = rand(1000); B = gpuArray(A);
>> timeit(@()circshift(A,[10 10]))
ans =
      0.0018
>> gputimeit(@()circshift(B,[10 10]))
ans =
      0.0020
>> A = rand(5000); B = gpuArray(A);
>> timeit(@()circshift(A,[10 10]))
ans =
      0.0266
>> gputimeit(@()circshift(B,[10 10]))
ans =
      0.0053
I'm not sure if I can advise how to improve it without writing your own version. Usually if you want to shift the data in a simple way you can reimplement it as some indexing and concatenation operations, which may well turn out to be faster for your particular use case.
Nevertheless I'll treat this as a request to improve the performance of circshift on the GPU.
Related Question