MATLAB: Out of memory: array too long

infinite rampreallocate out of memory

Hello, I'd like to run function pdist of Matlab on an array whose the length is N*(N-1)/2 where N=340000. Matlab is out of memory to preallocate this array. Could anyone give me a solution please? Kind regards, Winn

Best Answer

That's only going to require 57.8 gigabytes of memory for a single column. And that's the end result. Surely there'll be some large intermediate arrays as well.
I'd recommend downsampling or chunking up the calls.
Do you need every pairwise distance? Are you looking for something specific? What's the end goal?
Related Question