MATLAB: How working on a matrix with a large number of rows

Question

Hello everybody! I have a very large array, about 80000 rows and 3 columns. I need to calculate the distances between each line and the others.

I tried to calculate the possible combinations with the function nchoosek , but does not: out of memory

with a loop on the array takes too long …

how can I do?

Answer 1

try this:

Assuming X is storing your coordinates, (1 point per row) and (x,y,z) are the three columns, use pdist as follows:

d=pdist(X);

or if you are running out of memory:

d=pdist(single(X));

if you still go out of memory then you need bigger machine

Using pdist doesn't require you to calculate possible combination manually hence you save some memory there. Your original data 80000x3 at double precision takes less than 2MB of memory; so, I wouldn't worry about that loading it in memory. But your results in single precision would take about 12GB. Double precision make that double.