MATLAB: How could MATLAB store large scale of data.

large-scale

Hi, All,
I have to use a dataset as large as ones(3^N1,3^N2,10^3), where N1 and N2 can be 7~14. However, matlab will report "Out of memory" even for ones(3^14,1000). Is there a way to let matlab store such a large array.
I can split my data to several parts and store the sub-data. However that is very inconvenient.
I wonder if MATLAB can extend the limit, and handle large scale data. Thanks.

Best Answer

If you are able to use 8 bits per value, you could use
ones(3^N1, 3^N2, 10^3, 'uint8')  %or 'int8' as appropriate.
That would occupy 4.45 gigabytes, which would be too much for any 32 bit version of MATLAB (32 bits is 4 gigabytes of address space.) It would, however, be feasible with a 64 bit MATLAB with as little as 8 gigabytes of RAM.
If you need to use double precision, then that occupies 8 bytes per location, and so requires 35.64 gigabytes for the array. You would need a 64 bit MATLAB for sure, and you would need probably 40 or so gigabytes of RAM. If that much RAM is not feasible for you, then using a double precision array of that size is not feasible for you.
This is not a matter of how MATLAB uses its memory: this is just a matter of having enough memory to store the array at all.
Note: your options are different if the array is reasonably sparse.
Related Question